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SURVEILLANCE  OE  INFORMATION  TECHNOLOGY 
PERFORMANCE-BASED  SERVICE  ACQUISITIONS  (PBSAS) 


ABSTRACT 


There  has  been  an  increased  emphasis  on  utilization  Perfonnance  Based  Services 
Acquisitions  (PBSA)  as  a  means  for  organizations  to  cut  costs  and  to  become  more 
efficient.  According  to  GAO  report  05-274  nearly  half  of  an  organization’s  yearly  budget 
is  being  spent  on  services.  Consequently,  DoD  must  ensure  that  contractors  perfonning 
these  services  are  being  effectively  monitored.  This  will  ensure  that  required  services  are 
being  performed  according  to  contract  terms.  Based  on  this  GAO  report,  DoD  is  not 
adequately  performing  contractor  surveillance.  (GAO  Report  No.  05-274,  2005) 

This  can  be  accomplished  by  the  establishing  a  PBSA  team  from  the  onset  of  the 
acquisition  and  ensuring  that  the  technical  team  members  also  perform  the  contractor 
surveillance.  To  join  the  team,  each  person  must  have  received  training  on  PBSA  as  a 
condition  of  membership.  Furthermore,  team  members  should  have  access  to  lesson 
learned  from  previous  and/or  current  PBSA  experiences.  (Seven  Steps  to  PBSA,  2005) 

In  addition,  the  PBSA  should  ensure  that  FAR  1.102  guiding  principles  are  used 
when  developing  as  well  as  monitoring  contractor  performance.  (FAR  1.102) 

The  purpose  of  this  joint  applied  project  is  to  examine  IT  performance  based 
service  acquisitions  and  assess  what  performance  measurements  are  being  used  to  gauge 
the  contractor’s  effectiveness.  This  research  project  will  provide  a  literary  review  of 
Performance  Based  Contracts  (PBCs),  Performance  Based  Service  Contracts  (PBSC), 
Perfonnance  Based  Service  Acquisition  (PBSA)  and  related  topics.  This  review  will  be 
conducted  using  various  Governmental  legislative  and  regulatory  guidance  and  best 
industry  practices.  The  research  will  look  at  incentives  and  remedies  use  to  motivate  a 
contractor  to  succeed  performance  expectations.  The  research  will  include  a  gathering  of 
survey  infonnation  from  contracting  officers,  Directors  of  Information  Management 
offices,  Information  Technology  (IT)  technical  personnel  and  TACOM  IT  contractors. 
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This  information  will  then  provide  a  comparative  analysis  for  recommendations  and 
conclusions.  The  information  will  then  be  analyzed  to  aid  in  lessons  learned  processes, 
guidance,  and  policy  to  aid  other  acquisition  personnel  with  any  performance  based 
service  acquisitions.  To  conclude,  this  joint  applied  project  will  include  creating  and 
posting  performance  measurement  guidance  for  the  TACOM  Community  website  to 
provide  all  contracting  personnel  with  the  means  to  develop  performance  measures  for  IT 
performance  based  service  contracts. 
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I.  BACKGROUND 


A.  INTRODUCTION 

The  Department  of  Defense  (DoD)  is  starting  to  realize  that  surveillance  of 
service  acquisitions  needs  to  improve  significantly.  Even  though  DoD  has  taken  steps  to 
improve  general  oversight  and  management  of  service  contracts  through  the  National 
Defense  Authorization  Act  for  Fiscal  Year  2002,  these  efforts  have  done  very  little  to 
improve  service  contract  surveillance  (Government  Accountability  Office  (GAO)  Report 
05-274,  2005). 

Purchasing  services  from  the  private  sector  is  not  a  new  concept.  Outsourcing, 
which  is  now  referred  to  as  “Competitive  Sourcing”,  was  originally  directed  in  the  Office 
of  Management  and  Budget  (OMB)  Circular  No.  A-76  issued  in  1966.  However,  it  is 
interesting  to  note  that  the  commercial  market  still  refers  to  this  type  of  competition  as 
“outsourcing.”  According  to  this  circular,  the  Government  will  not  “start  or  carry  on  any 
activity  to  provide  a  commercial  product  or  service  if  the  product  or  service  can  be 
procured  more  economically  from  a  commercial  source.”  Information  Technology  (IT) 
services  have  been  long  considered  a  function  which  private  industry  can  perform. 
Consequently,  IT  services  have  become  a  target  for  A-76  studies  a  means  for  federal 
agencies  procure  services  more  economically  thereby  reducing  their  operational  costs. 
By  using  competitive  sourcing  processes  to  obtain  IT  services,  Government  agencies  are 
allowed  to  concentrate  on  their  core  mission  as  well  reducing  their  overall  operational 
costs.  Even  though  the  requirement  to  outsource  services,  like  IT  services,  has  been  a 
requirement  since  1966,  it  has  just  been  in  the  last  few  years  that  purchases  of  services 
have  seen  a  significant  increase.  Agencies  are  being  pushed  to  use  performance-based 
service  contracting  (PBSCs)  methods,  to  the  maximum  extent  possible,  when  developing 
service  contracts  (Office  of  Federal  Procurement  Policy  (OFPP)  Letter  91-2,  Service 
Contracting  and  OFPP  report,  2003).  The  use  of  service  based  contracting  methods  is  a 
whole  new  arena  for  DOD  acquisition  personnel.  As  a  result  of  these  changes, 
surveillance  of  service  contracts  has  become  more  important  to  ensure  that  the 
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Government  agencies  receive  services  they  require  in  a  timely  and  quality  manner  (GAO 
Report  05-0274,  2005).  These  developments  will  be  discussed  as  part  of  this  research. 

The  purpose  of  this  Joint  Applied  Project  professional  report  is  to  analyze  IT 
performance  based  service  acquisitions  within  the  U.S.  Anny  Tank  Automotive  and 
Armaments  Command  (TACOM)  and  to  evaluate  what  performance  measurements  are 
being  used  to  gauge  the  contractor’s  effectiveness  against  the  performance  measures 
stated  in  the  contract.  Additionally,  this  report  will  also  address  incentives,  as  well  as 
penalties,  used  to  incentivize  the  contractor  to  provide  the  level  of  service  envisioned  by 
the  Government.  In  conclusion,  the  report  will  analyze  how  development  of  perfonnance 
measures,  incentives,  and  penalties  affect  the  ability  of  the  surveillance  team  to 
adequately  administer  the  contract. 

The  methodology  of  this  research  will  include  several  steps.  Literary  review  will 
discuss  topics  such  as  Perfonnance  Based  Contracts  (PBCs),  Performance  Based  Service 
Contracts  (PBSCs),  Performance  Based  Service  Acquisitions  (PBSAs),  Quality 
Assurance  Surveillance  Plans  (QASPs),  Performance  Work  Statements  (PWSs),  and 
Statement  of  Objectives  (SOOs),  incentives,  performance  measurements  and  metrics. 
This  literary  review  will  be  conducted  to  include  government  and  commercial  sources. 
Surveys  and  interviews  will  be  perfonned  and  include  participants  from  both  government 
and  private  industry.  The  results  of  the  surveys  and  interviews  will  be  included  in 
Chapter  III.  The  results  of  the  surveys  and  interviews  will  be  analyzed  and  a  comparison 
of  these  results,  with  the  information  from  Chapter  II,  Literary  Review,  will  be  included 
in  Chapter  IV.  Recommendations  and  areas  of  further  research  will  be  included  in  this 
research  project  in  Chapter  V.  The  above  information  will  provide  the  PBSA  Team  with 
lessons  learned  and  commercial  best  practices  necessary  to  ensure  successful  contract 
surveillance. 

B.  TERMS  AND  DEFINITIONS  AS  DEFINED  IN  FAR,  DFARS,  ETC. 

Performance-based  contracting,  as  defined  in  the  Federal  Acquisition 
Regulation  (FAR),  is  a  means  to  structure  all  aspects  of  an  acquisition  around  the  purpose 
of  the  work  to  be  perfonned  with  the  contract  requirements  to  be  set  forth  in  clear, 
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specific,  and  objective  terms  with  measurable  outcomes  (FAR  2.101).  This  topic  will  be 
discussed  further  in  Chapter  II  Literary  Review. 

Performance-Based  Service  Acquisition  (PBS A)  is  defined  as  structuring  all 
aspects  of  an  acquisition  around  the  work  to  be  performed  with  the  contract  requirements 
set  forth  in  clear,  specific,  and  objective  terms  with  measurable  outcomes  rather  than 
direct  perfonnance  processes.  Simply  stated,  “it  is  a  method  for  acquiring  what  is 
required  and  places  the  responsibility  for  how  it  is  accomplished  with  the 
contractor. ’’(Guidebook  for  Performance-Based  Service  Acquisition  (PBSA)  in  the  DoD, 
2001)  The  use  of  the  word  "acquisition"  rather  than  "contracting"  reflects  the  broader 
scope  (from  mission  planning  to  contract  performance  management)  and  broader 
community  (including  program  offices)  that  are  a  necessary  part  of  the  process 
(Guidebook  for  PBSA  in  the  DoD,  2001). 

C.  RESEARCH  OBJECTIVE 

The  objective  of  this  research  is  to  gauge  the  effectiveness  of  existing  IT  PBS  As 
and  use  the  lessons  learned  to  develop  and  post  performance  measurement  guidance  on 
the  TACOM  Community  website  to  provide  all  contracting  personnel  with  a  means  to 
develop  performance  measures  for  IT  PBSAs. 

D.  RESEARCH  QUESTIONS 

The  research  questions  to  be  answered  in  this  Joint  Applied  Project  are: 

1.  To  what  extent  are  performance-based  measures  being  used  to  manage 
TACOM  IT  PBSAs? 

2.  How  clear  and  applicable  are  PBSA  contracting  policies  and  procedures, 
e.g.,  statements  of  work  (SOWs),  performance  work  statements  (PWS),  and  statements  of 
objectives  (SOOs)? 

3.  What  are  the  incentives  and  penalties  affecting  PBSAs? 

4.  What  are  ways  to  improve  U.S.  Army,  TACOM-RI,  PBSA  including 
methods  promulgating  web-based  guidance  to  practitioners? 
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E.  SCOPE  (BENEFIT  TO  TACOM  COMMUNITY),  LIMITATIONS,  AND 

ASSUMPTIONS 

This  research  project  will  provide  the  TACOM  community  with 
recommendations  and  guidance  related  to  developing  successful  performance 
measurements  to  use  with  IT  Performance  Based  Service  Acquisitions.  In  addition,  this 
site  will  include  lessons  learned  and  frequently  asked  questions  regarding  the  use  of 
incentives  (positive  and  negative);  it  will  provide  websites,  Government  documents,  and 
commercial  sources  available  to  assist  contracting  personnel  in  monitoring  and  accessing 
contractor  perfonnance;  and  it  will  provide  samples  of  performance  measures  and 
incentives  that  have  been  successfully  used  in  past  and  current  contracts. 

This  project  is  limited  to  review  of  IT  PBSAs  within  the  TACOM  community  that 
have  successfully  used  competitive  sourcing  techniques  or  are  follow-on  to  a  competitive 
sourcing  PBSA. 

Assumptions  being  made  are  that  current  IT  PBSAs  lack  effective  performance 
measurements,  metrics,  and  incentives  to  effectively  monitor  of  contract  perfonnance. 

F.  METHODOLOGY 

The  methodology  used  for  the  purposes  of  this  thesis  consists  of  literary  reviews 
of  Performance  Base  Contracts  (PBCs),  Perfonnance  Based  Service  Contracts  (PBSCs), 
Perfonnance  Based  Service  Acquisitions  (PBSAs),  Information  Technology  (IT)  PBSAs, 
and  surveillance  of  IT  PBSAs  to  determine  if  adequate  surveillance  is  cunently  being 
accomplished  on  these  types  of  contracts.  The  literary  review  will  include  Government 
Legislation,  DoD  Regulations,  DoD  policy  and  guidance,  interim  policy,  and  regulations; 
best  practices;  and  best  commercial  practices  regarding  service  contracts  and/or 
acquisitions. 

The  report  will  also  will  attempt  to  obtain  additional  insight  relating  to 
management  of  PBSAs  from  personnel  within  the  TACOM  community.  These 
individuals  are  responsible  for  managing  and  monitoring  IT  PBSAs.  This  will  be 
accomplished  with  completion  of  surveys  or  conducting  interviews  with  Government  and 
Contractor  Project  Managers,  Directors  of  Information  Management  (DOIMs),  Procuring 
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Contracting  Officers,  Government  IT  technical  personnel,  TACOM  contractors,  and  other 
1102  series  acquisition  personnel.  After  receipt  of  the  questionnaires  and  completion  of 
the  interviews,  this  information  will  be  compiled  and  analyzed  to  provide  information  on 
what  is  actually  happening  in  the  TACOM  community  regarding  surveillance  of  IT 
PBSAs.  Based  on  the  findings  from  the  surveys  received,  this  report  will  cover  any 
interviews  conducted  to  obtain  further  infonnation. 

The  report  will  conclude  with  a  comparison  and  contrast  on  how  surveillance 
should  be  perfonned  with  what  is  actually  being  conducted.  Finally,  a  template  will  be 
developed  to  provide  guidance  on  the  development  of  metrics  and  incentives  to  assist  the 
PBSA  Team  in  creation  of  PWSs,  SOOs,  QASPs,  and  the  types  of  incentives  necessary  to 
incentivize  contractor  performance. 

G.  BENEFIT  OF  RESEARCH 

Anticipated  benefits  of  our  research  will  include  creating  lessons  learned  and 
posting  performance  measurement  guidance  for  the  TACOM  community,  which  will 
provide  acquisition  personnel  with  the  means  to  develop  performance  measures  to  ensure 
proper  surveillance  of  IT  PBSA  contracts. 

H.  ORGANIZATION  OF  THESIS 

This  report  is  organized  into  five  chapters.  Chapter  I  provided  an  overview  of  the 
research  project,  objectives,  and  methodology.  Chapter  II  includes  a  literary  review  to 
include  topics  such  as  PBC,  PBSCs,  and  PBSAs.  Along  with  these  topics,  Chapter  II  will 
include  definitions  for  the  acquisition  techniques  and  for  related  tasks  that  are  required 
with  these  types  of  acquisitions.  The  literary  review  will  be  conducted  using  Government 
Legislation,  regulations,  DoD  guidance,  and  commercial  best  practices.  Chapter  III  will 
provide  a  real-world  comparison  based  on  surveys  and  interviews  conducted  from  this 
research  project.  Additionally,  Chapter  III  will  also  include  a  list  of  the  questions 
contained  in  the  survey  and  interviews,  and  the  selection  of  the  respondents  and 
interviewees.  Chapter  IV  will  compare  and  contrast  results  of  the  literary  review  with  the 
results  of  the  questionnaire/interviews.  Chapter  IV  will  summarize  the  results  of  the 
analysis  obtained  in  Chapter  III.  Chapter  V  provides  overall  conclusions, 
recommendations,  and  recommended  areas  for  further  research. 
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I.  SUMMARY 

In  conclusion.  Chapter  I  of  this  research  project  states  the  objective  of  this  project, 
which  is  to  determine  if  IT  PBSAs  are  being  effectively  measured  and,  if  so,  what  lessons 
have  been  learned.  Included  in  this  chapter  are  the  related  terms  and  definitions  relating 
to  PBSAs.  These  tenns  and  others  are  further  discussed  in  Chapter  II  -  Literary  review. 
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II.  LITERARY  REVIEW 


A.  INTRODUCTION 

The  literary  that  will  be  discussed  in  this  chapter  will  include  topics  such  as 
Performance  Based  Contracts  (PBCs),  Performance  Based  Service  Contracts  (PBSCs), 
and  Performance  Based  Service  Acquisitions  (PBSAs).  This  will  involve  distinguishing 
between  the  PBCs,  PBSCs,  and  PBSAs.  The  research  will  also  include  components  and 
objectives  of  PBSA;  therefore,  additional  terms  will  be  discussed.  The  literary  review 
will  be  conducted  using  Government  legislation  and  regulations,  DoD  guidance,  and 
commercial  best  practices.  In  conducting  this  process,  the  literary  review  will  discuss 
both  Government  regulatory  guidance  and  industry  best  practices. 

B.  ELEMENTS  OF  PERFORMANCE  BASED  ACQUISITIONS 

The  DoD  PBSA  Guidebook  states  that  in  order  to  be  considered  performance- 
based,  an  acquisition  should  contain  the  following  elements: 

A  performance  work  statement  (PWS)  describes  the  acquisition  requirement  in 
tenns  of  measurable  outcomes  rather  than  telling  a  contractor  how  to  perform  the 
supplies  and/or  services  required. 

Measurable  performance  standards  or  metrics  are  used  to  determine  whether 
desired  outcomes  have  been  met.  Consequently,  the  standards  or  metrics  help  to  define 
what  will  be  considered  acceptable  performance  under  any  resulting  contract. 

Incentives  and  Remedies  provide  a  means  to  address  acceptable  and 
unacceptable  contractor  performance.  Not  all  acquisitions  contain  incentives  but  the  team 
should  be  encouraged  to  include  them  whenever  possible.  Incentives  are  included  to 
motivate  the  contractor  to  exceed  performance  standards.  Additionally,  this  guidebook 
states  “Remedies  and  incentives  complement  each  other.” 

A  performance  Assessment  plan  describes  how  contractor  performance  is 

monitored  and  assessed.  This  plan  can  be  written  by  the  Government  as  part  of  the  PWS 

or  can  be  a  requirement  of  the  solicitation.  Therefore,  the  contractor  can  include  it  as  part 

of  their  proposal.  If  the  Government  writes  the  Perfonnance  Assessment  Plan,  it  is  called 
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a  Quality  Assurance  Surveillance  Plan  (QASP)  while  if  contractor  provides  the 
Performance  Acceptance  Plan  as  part  of  their  proposal,  it  is  known  as  Quality  Assurance 
Plan  (QAP)  (PBSA  in  the  DoD,  December  2000). 

Although  the  terms  PBC,  PBSC,  and  PBSA  may  mean  different  contracting 
methods  for  some  people,  for  the  purposes  of  this  research  project,  the  terms  PBC,  PBSC, 
and  PBSA  will  be  used  interchangeably.  Whether  referred  to  as  Performance-Based 
Service  Contracting  (PBSC),  Performance-Based  Service  Acquisition  (PBSA), 
Performance-Based  Acquisition  (PBA),  or  Performance-Based  Contracting  (PBC),  the 
objective  of  improved  performance  remains  the  same.  No  longer  the  exception  to 
traditional  contracting,  these  perfonnance-based  approaches  simply  make  good  business 
sense  (KnowNet,  2005). 

The  following  tenns  and  definitions  will  be  used  during  the  discussion  of 
contractor  surveillance: 

Acceptable  Quality  Level  (AQL)  is  the  variance  from  a  performance  standard 
that  a  contractor  is  allowed,  before  the  Government  rejects  its  services.  Generally 
speaking,  a  contractor  will  be  given  an  opportunity  to  correct  non-confonning  services  if 
it  can  be  accomplished  within  the  required  delivery  schedule  (KnowNet,  2005). 

The  Quality  Assurance  Evaluator  (QAE)  is  a  technical  or  subject  matter  expert 
that  is  qualified  to  perform  the  quality  assurance  functions  required  under  the  PBSA 
(KnowNet,  2005). 

A  Contracting  Officer's  Representative  (COR)  is  a  government  employee 
whom  is  appointed  in  writing  and  delegated  by  the  Contracting  Officer  to  perform 
specific  contract-related  decisions.  They  can  also  be  referred  to  as  Contracting  Officer's 
Technical  Representative  (COTR)  (KnowNet,  2005). 

C.  GOVERNMENT  LEGISLATION 

1.  Government  Performance  and  Results  Act  (GPRA)  of  1993 

The  rationale  of  GPRA  was  to  make  agencies  accountable  for  program 
performance.  GPRA  was  passed  in  response  to  concerns  regarding  Federal  waste  and 
inefficiencies.  Like  PBCs,  agencies  are  required  to  establish  and  manage  “mission- 
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related  performance  goals  and  objectives”  (KnowNet,  2005).  In  essence,  this  act  has  laid 
the  groundwork  for  acquisition  teams  to  work  towards  result  driven  acquisitions  instead 
of  telling  the  contractor  how  to  perform  the  requirement.  Consequently,  PBSAs  can  help 
to  support  the  goals  required  by  GPRA,  as  well  as  to  achieve  the  agency’s  mission.  The 
PBSA  team  can  accomplish  this  by  clearly  describing  the  acquisition  as  it  relates  to  its 
mission,  thereby  setting  the  contractor’s  goals  to  be  in  sync  with  that  of  the  organization. 
(KnowNet,  2005) 

2.  Clinger  Cohen  Act  (CCA)  of  1996 

The  CCA  states  that  Government  IT  organizations  should  operate  like  a  business; 
therefore,  these  organizations  must  look  for  ways  to  streamline  their  processes  and  to  run 
more  efficiently.  This  act  states  the  “acquisition  technology  planning  and  management” 
must  be  treated  as  a  "capital  investment."  Therefore,  organizations  cannot  and  should  not 
acquire  hardware  and  software  systems  without  first  looking  at  all  the  facets  of  the 
acquisition  and  make  a  sound  business  decision  before  pursing  any  acquisition. 

Just  like  PBSA  and  the  GPRA,  the  CCA  requires  the  IT  organization  to  “establish 
goals  for  improving  efficiencies  and  effectiveness”  for  day-to-day  operations.  These 
organizations  must  also  “ensure  the  established  performance  measurement”  can  produce 
the  desired  outcomes  and  are  aligned  with  the  organization’s  mission. 

D.  OFFICE  OF  FEDERAL  PROCUREMENT  POLICY  (OFPP) 

The  OFPP  recognized  the  need  to  provide  guidance  to  the  acquisition  workforce 
regarding  PBSAs.  Accordingly,  the  OFPP  put  together  an  interagency  working  group  in 
April  2002  in  order  to  evaluate  and  provide  acquisition  personnel  with  a  better 
understanding  of  PBSAs.  This  team  reviewed  several  internal  documents  which  included 
OFPP  “Guide  to  Best  Practice  for  Performance-Based  Service  Contracts”,  1998,  and 
OFPP  Pamphlet  No.  4,  “A  Guide  to  Writing  and  Administering  Performance  Statement 
of  Work  for  Service  Contracts”,  1980.  In  addition,  the  team  reviewed  a  report  prepared 
by  Dr.  Lawrence  Martin,  Pricewaterhouse  Coopers  Endowment  for  the  Business  of 
Government  title  “Making  Perfonnance  Based  Contract  Perform:  What  the  Federal 
Government  Can  Learn  from  State  and  Local  Governments”,  which  will  be  discussed 
later  in  this  chapter.  The  task  force  also  reviewed  OFPP’s  “Solicitation/Contract  Task 
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Order  Review  Checklist,  dated  August  1997,  and  the  latest  revision  of  “The  Steps  to 
Performance  Based  Service  Acquisition  Guide”. 

Based  on  the  results  of  this  working  group,  a  report  was  issued  titled, 
“Performance-Based  Acquisition  Contracting  for  the  Future”,  dated  July  2003.  The 
report  recommended  that  FAR  Part  2  be  revised  to  include  definitions  for  performance 
work  statement  (PWS),  quality  surveillance  plan  (QAP),  statement  of  objectives  (SOOs), 
etc.  The  PWS  would  be  used  to  identify  the  technical,  functional  and  performance 
characteristics  of  an  agency  requirement.  A  QASP  would  be  redefined  to  mean  a  plan  for 
assessing  the  contractor’s  perfonnance,  which  would  ensure  the  Government’s 
performance  objectives  were  met  and  were  in  compliance  with  the  terms  and  conditions 
of  the  contract. 

As  already  defined,  a  SOO  is  alternative  to  the  PWS.  However,  the  team 
recommended  that  the  SOO  should  provide  a  summary  of  key  agency  goals  and/or 
outcomes.  This  would  allow  the  contractor  to  draft,  for  government  review,  a  PWS  to  be 
incorporated  into  any  resulting  acquisition. 

Finally,  the  SOW,  much  like  the  PWS,  would  be  defined  as  a  statement  that 
describes  the  Government’s  requirements  in  a  clear  and  concise  manner. 

The  Interagency  Working  Group,  chartered  by  the  OFPP,  also  recommended  that 
the  term  Performance  Based  Service  Acquisition  (PBSA)  be  used  “to  provide  common 
tenninology  throughout  the  Government”.  In  addition,  this  committee  recommended  that 
FAR  37.601  (a)  be  changed  to  read  as  follows:  “The  principal  objective  of  performance- 
based  service  acquisition  (PBSAs)  is  to  express  government  needs  in  terms  of  required 
performance  objectives,  rather  than  the  method  of  perfonnance,  to  encourage  industry- 
driven,  competitive  solutions.”  (OFPP  Interagency  Working  Group,  2004) 

As  result  of  this  task  force,  OFPP  issued  memorandum  Subject:  “Increasing  the 
Use  of  Performance  Based  Service  Acquisitions”,  dated  September  4,  2004,  and 
rescinded  OFPP’s  “1998  Guide  to  Best  Practices  for  Performanced-Based  Service 
contracting”.  This  memorandum  stated  that  agencies  should  use  the  “Seven  Steps  to 
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Performance-Based  Service  Acquisition  Guide,  available  at  www.acqnet.gov  when 
developing  PBSAs.  (OFPP  Interagency  Working  Group,  2004) 

E.  FEDERAL  ACQUISITION  REGULATION  (FAR) 

The  FAR  provides  a  definition  for  PBC;  however,  it  fails  to  properly  define  the 
differences  between  PBC  and  PBSA,  as  well  as  to  include  definitions  of  other  PBSA 
terms. 

As  stated  in  Chapter  One,  FAR  Part  2  defines  “Performance-based  contracting”  as 
the  means  of  structuring  all  aspects  of  an  acquisition  around  the  purpose  of  the  work  to  be 
performed  with  the  contract  requirements  set  forth  in  clear,  specific,  and  objective  terms 
with  measurable  outcomes  as  opposed  to  how  the  work  is  to  be  performed  or  by  broad 
and  imprecise  statements  of  work.  (FAR  2.101) 

Additionally,  FAR  37.101,  Definitions,  defines  a  service  contract  as  “a  contract 
that  directly  engages  the  time  and  effort  of  a  contractor  whose  primary  purpose  is  to 
perform  an  identifiable  task  rather  than  to  furnish  an  end  item  of  supply.”  (FAR  37.101) 

The  policy  at  FAR  37.102  states  that  Performance  Based  contracts  are  the 
preferred  method  of  contracting  for  services.  Moreover,  these  contracts  should  be 
awarded  under  a  firm  fixed  price  whenever  feasible  before  another  type  of  contract  is 
used.  Consequently,  there  seems  to  be  a  need  to  provide  and/or  clarify  what  the 
difference  is  between  PBCs  and  PBSAs.  Whereas  a  PBC  can  be  used  to  acquire 
logistical  support  to  include  both  hardware  and  services,  such  as  provided  in  a  contract 
for  total  contractor  logistical  support  or  supply  chain  management,  PBSAs  are  needed  to 
provide  services  such  as  base  operation  support,  guard  services,  environmental 
remediation  actions,  janitorial,  and  IT  support  services. 

Additionally,  for  any  PBSA  to  be  successful  the  team  must  adhere  to  FAR 
guidance  which  states  the  team  must  be  established  from  “...beginning  with  the  customer 
and  ending  with  the  contractor  of  the  product  or  service.  By  identifying  the  team 
members  in  this  manner,  teamwork,  unity  of  purpose,  and  open  communication  among 
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the  members  of  the  Team  in  sharing  the  vision  and  achieving  the  goal  of  the  System  are 
encouraged.  Individual  team  members  will  participate  in  the  acquisition  process  at  the 
appropriate  time.”  (FAR  1.102-3) 

F.  DEFENSE  FEDERAL  ACQUISITION  REGULATIONS  (DFARS) 

A  review  of  the  DFARs  did  not  provide  definitions  for  performance  based 
contracts  nor  did  it  provide  significant  information  to  assist  the  team  with  the  formulation 
and  surveillance  of  any  service  acquisitions. 

Surveillance  is  required  by  DFARS  for  all  PBSAs;  however,  DFARS  does  not 
specify  how  surveillance  will  be  performed.  Depending  on  the  contract,  organization 
surveillance  ranges  from  formal  written  assessments,  which  are  conducted  monthly, 
quarterly,  semi-annually,  or  annually,  to  random  inspection  of  work  orders,  tickets,  etc. 

G.  DEFENSE  ACQUISITION  GUIDEBOOK  (DAG) 

The  Defense  Acquisition  Guidebook  (DAG)  was  created  to  provide  the 
acquisition  workforce  with  guidance  regarding  the  DoD  Instruction  5000.1  and  5000.2. 
The  DAG  contains  eleven  (11)  chapters;  chapter  2  contains  information  regarding 
acquisition  program  goals.  Additionally,  chapter  seven  (7)  contains  information  which 
informs  the  team  of  the  statutory  and  regulatory  requirements  for  acquiring  IT.  This 
chapter  also  provides  information  regarding  the  Clinger-Cohen  Act,  and  it  relates  to  the 
IT  acquisitions.  Consequently,  this  guidebook  can  provide  the  team  and  the  acquisition 
personnel  with  reference  guidance  to  enable  them  to  develop  their  acquisition  strategy 
(DAG,  2004). 

H.  DOD  DIRECTIVE  5000  SERIES  GUIDANCE 

While  the  DoD  Directive  5000  does  not  specifically  provide  information  on 
performance  based  service  contracts,  it  does  provide  the  team  with  the  conceptual 
framework  for  development  of  the  acquisition.  This  framework  indicates  that,  as  with 
any  acquisition,  the  team  should  be  comprised  of  technical  personnel,  acquisition 
personnel  and  contractor  personnel,  depending  on  the  acquisition. 


12 


The  PBS  A  team  should  be  set  up  from  the  beginning  and  all  team  members 
should  be  part  of  the  market  research,  acquisition  strategy  and  remain  with  the  team 
throughout  the  life  of  the  contract. 

1.  DoD  Instruction  5000.2 

A  review  of  the  DoD  Directive  5000.2  indicates  that  Integrated  Product  Teams 
(IPTs)  shall  be  used  to  communicate  with  the  stakeholders  throughout  the  acquisition 
process.  It  further  states  that  the  Program  Manager  has  the  ultimate  responsibility  of 
coordinating  these  efforts  (DoDI  5000.2,  2004). 

Additionally,  this  directive  also  states  that  the  Perfonnance  Based  Acquisitions 
(PBAs)  shall  be  used  to  the  maximum  extent  possible.  The  intent  is  “To  maximize 
competition,  innovation,  and  interoperability,  and  to  enable  greater  flexibility  in 
capitalizing  on  commercial  technologies  to  reduce  costs,  acquisition  managers  shall 
consider  and  use  performance-based  strategies  for  acquiring  and  sustaining  products  and 
services  whenever  feasible.”  Therefore,  the  acquisition  requirements  “shall  be  stated  in 
performance  terms.”  Thus,  it  encourages  the  team  to  constantly  seek  to  the  use  of 
performance  based  strategies  whenever  possible  (DoDI  5000.2,  2004  Enclosure  1). 

I.  GUIDELINES  -  SEVEN  STEPS  TO  PERFORMANCE  BASED  SERVICE 

ACQUISITION 

This  web-based  guide  describes  performance  based  service  acquisition  using 
seven  steps.  The  seven  steps  are  described  below: 

1.  Establish  the  Team 

An  Integrated  or  Perfonnance  Based  Service  Acquisition  (PBSA)  Team  must  be 
established.  This  team  should  assist  in  the  acquisition  planning,  the  solicitation 
development,  evaluation,  and  award  phase  of  the  acquisition.  After  contract  award,  the 
PBSA  team  will  be  closely  involved  in  the  surveillance  of  the  resulting  contractor’s 
performance. 

2.  Describe  the  Problem 

This  step  requires  a  description  of  the  problem  that  needs  to  be  solved.  GPRA 
plays  a  part  in  this  step  as  it  requires  the  agency  to  establish  and  “manage”  to  mission- 
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related  performance  goals  and  objectives.  Additionally,  the  team  can  describe  the 
problem  is  terms  of  a  PWS  or  a  SOO. 

3.  Examine  Private-Sector  &  Public  Sector  Solutions 

Under  this  step,  the  team  should  examine  both  government  and  private  sector 
solutions.  This  is  otherwise  known  as  Market  Research.  Market  research  is  a  good 
source  for  finding  the  appropriate  contract  type  to  fit  a  specific  situation.  It  is  up  to  the 
PBSC  team  to  make  a  good  business  decision  and  utilize  negotiations  with  the  contractor 
to  determine  the  right  contract  type. 

In  order  to  make  a  “good”  business  decision,  the  PBSC  team  must  be  clear  of  the 
pros  and  cons  associated  with  each  type  of  contract  and  what  type  of  incentives  and 
disincentives  they  provide. 

A  fixed  price  contract  requires  the  contractor  to  be  responsible  for  all  costs, 
whether  the  contractor  realizes  a  profit  or  loss  under  the  contract  is  determined  by  his 
performance. 

A  fixed  price  incentive  contract  allows  for  the  final  contract  price  and  profit  to  be 
calculated  based  on  a  formula  stated  in  the  contract  that  relates  the  final  negotiated  costs 
to  the  target  cost. 

A  fixed  price  contract  with  award  fee  is  used  when  contract  perfonnance  cannot 
be  measured  objectively.  This  is  a  way  to  motivate  the  contractor  when  other  incentives 
are  not  appropriate. 

Cost-reimbursement  incentive  contracts  are  used  when  fixed-price  contracts  are 
not  appropriate  because  there  is  uncertainty  regarding  probable  costs.  These  types  of 
contracts  can  be  cost  plus  incentive  fee  or  cost  plus  award  fee. 

A  cost-plus  fixed  fee  contract  provides  the  contractor  with  minimal  responsibility 
to  control  costs.  The  contractor  is  reimbursed  for  all  allowable  and  allocable  costs  and 
their  profit  (fee)  is  negotiated  and  fixed. 

The  current  regulatory  guidance  provides  a  preference  to  utilize  firm  fixed  price 
contracts  for  IT  PBSAs.  In  the  past,  contracts  that  provide  for  support  services  have  been 
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awarded  as  cost  reimbursement  type  contracts,  and  the  contractor  community  is  very 
reluctant  to  change  from  reimbursement  contracts  to  firm  fixed  price  contracts. 

The  argument  for  firm  fixed  price  contracts  for  IT  PBS  As  is  these  types  of 
contracts  state  desired  outcomes  and  identifies  how  performance  will  be  measured.  In  a 
contract  where  performance  is  specified,  but  price  is  constrained,  it  is  possible  that 
performance  might  exceed  expectations. 

Please  note  the  contract  does  not  have  to  be  one  of  the  contract  types  listed  above; 
it  can  be  a  hybrid  contract  that  contains  both  fixed  price  and  cost  reimbursement 
performance  measures. 

4.  Develop  PWS  or  SOO 

Step  4  discusses  the  development  of  a  PWS  or  SOO.  While  a  PWS  provides  the 
contractor  with  desired  results  and  measurable,  relevant  standards,  a  SOO  provides  a 
prospective  contractor  with  the  desired  objectives.  A  contractor  then  submits  a  proposal 
in  the  form  of  a  PWS  for  the  Government  to  review.  As  KnowNet  states,  “a  SOO  is  an 
alternative  approach  to  using  a  PWS.”  A  SOO  may  ensure  the  contractor  can  meet  or 
exceed  the  desired  contract  perfonnance.  When  drafting  a  SOO  for  an  IT  PBSA,  the 
following  objectives  can  be  employed: 

Ensure  that  customer  support  is  available  24/7 

Ensure  that  email  services  are  operational  24/7 

This  allows  for  flexibility  of  the  contractor’s  workforce  to  cover  emergency  and 
after-hour  requirements. 

5.  Decide  How  to  Measure  &  Manage  Performance 

According  to  this  guide,  “this  step  establishes  the  strategy  of  managing  the 

contract  to  achieve  planned  perfonnance  objectives.  The  OFPP’s  “A  Guide  to  Best 

Practice  Performance  Based  Service  Contracting”,  dated  October  1998,  states  that  a  QAP 

defines  what  the  Government  must  do  to  ensure  that  the  contractor  performs  to  the  PWS. 

Moreover,  a  QAP  or  QASP  is  a  plan  for  which  the  Government  QAE  measures  the 

contractor’s  perfonnance  against  the  standards  contained  in  the  PWS.  This  plan  should 

be  clear,  concise,  and  inform  the  contractor  what  surveillance  methods  will  be  employed 
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in  any  resulting  contract.  The  QASP  should  focus  on  the  critical  tasks  contained  in  the 
PWS,  which  are  measurable  and  attainable.  It  tells  the  contractor  how  they  will  be  paid 
for  services  rendered  to  ensure  the  Government  receives  the  services  required. 

Consequently,  the  QASP  provides  the  Government  with  an  indication  of  what 
resources  are  needed  to  properly  administer  and  assess  the  contractor’s  performance. 
Even  so,  the  contractor  should  not  rely  solely  on  the  QASP  to  tell  them  how  to  perform 
any  required  services. 

In  developing  the  QASP,  the  Government  team  should  look  at  the  PWS  and 
decide  which  tasks  are  critical  and  worth  measuring.  It  is  better  to  choose  fewer  metrics 
which  are  meaningful  and  relevant  than  to  select  numerous  or  complex  metrics.  When 
selecting  the  metrics,  the  team  should  look  at  the  metrics  to  ensure  they  are  easy  to 
collect.  If  not,  they  may  not  be  achievable  and  are  not  worth  being  selected  as  a  metric. 
To  be  achievable  means  the  contractor  will  not  require  any  assistance  from  Government 
personnel  to  provide  the  required  services.  If  the  contractor  requires  Government 
assistance,  why  assess  or  include  this  metric  since  it  requires  both  parties  to  achieve  the 
metric.  Monitoring  the  contractor’s  performance  can  be  costly,  especially  if  the 
Government  chooses  to  monitor  the  contractor  for  100%  compliance.  The  question  that 
should  be  asked  before  you  require  100%  compliance  is  will  this  actually  be  required  to 
get  the  level  of  performance  we  need.  We  think  if  the  “right  contractor”  is  chosen,  100% 
compliance  will  not  be  necessary. 

As  stated  in  Step  6  of  the  7  steps  to  Perfonnance  Based  Service  Acquisitions,  the 
“right  contractor”  must  be  chosen  and  the  Government  should  build  a  working 
relationship  with  the  contractor  in  order  for  both  to  have  a  true  understanding  of  the  work 
required  under  the  contract. 

Additionally,  step  5  includes  development  of  the  contractual  language, 
establishment  of  the  contract  type,  incentives,  metrics,  etc. 

6.  Selecting  the  Right  Contractor 

Step  6  involves  the  developing  an  acquisition  strategy,  which  will  assist  the  team 
in  selecting  a  contractor  who  provides  the  best  value  for  a  given  service.  The  best  value 
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contractor  may  not  provide  the  lowest  price,  but  may  offer  a  better  approach  to  the 
requirement  or  may  offer  a  lower  risk  to  the  Government  due  to  a  higher  past 
performance  rating.  When  evaluating  each  contractor,  the  PBSA  team  must  ensure  each 
offeror  understands  PBSAs  and  has  the  experience/knowledge/expertise  to  perform  the 
services  required  by  the  solicitation. 

7.  Manage  the  Contractor’s  Performance 

This  step  is  where  “the  real  work”  of  the  acquisition  comes  into  play.  The  agency 
must  ensure  that  the  proper  resources  are  allocated  to  ensure  the  contractor  performs 
IAW  PWS.  The  Contracting  Officer  must  inform  the  team  of  their  responsibilities  and 
limitations  during  this  phase  to  avoid  any  unauthorized  actions.  This  step,  contract 
performance  “is  guided  far  less  by  law,  regulation,  and  policy  than  those  described  in  the 
preceding  steps.”  Consequently,  the  team  must  be  aware  of  the  terms  and  conditions  of 
the  contract  and  the  PWS  to  ensure  they  are  acting  within  their  authority  (Seven  Steps  to 
PBSA,  2005). 

J.  GENERAL  ACCOUNTABILITY  OFFICE  (GAO) 

The  Department  of  Defense  (DoD)  has  reported  that  DoD  agencies  need  to 
improve  surveillance  on  their  service  contracts.  In  the  GAO  report  titled,  “Contract 
Management,  Opportunities  to  Improve  Surveillance  on  Department  of  Defense  Service 
Contracts”(March  2005),”  GAO  states  that  the  problem  with  surveillance  revolves  around 
improperly  trained  service  personnel,  surveillance  personnel  not  being  assigned  to  the 
service  contract  by  the  date  of  contract  award,  surveillance  personnel  not  being  held 
accountable  for  their  duties,  incomplete  documentation  of  surveillance  personnel,  and 
contract  surveillance  taking  a  back  seat  to  awarding  contracts.  Additionally,  the  GAO 
report  states  that  these  types  of  contracts  lack  the  sufficient  reporting  requirements  to 
ensure  that  services  are  required  and  warranted.  The  report  also  states  that  without 
proper  documentation  and  surveillance,  DoD  is  at  risk  of  being  unable  to  identify  or 
correct  poor  performance  in  a  timely  manner.  In  conclusion,  GAO  recommended  that 
DoD  require  properly  trained  surveillance  personnel  be  assigned  at  the  onset  of  the 
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contract  award.  In  addition,  GAO  stated  that  the  surveillance  personnel  must  be  held 
accountable  regarding  their  duties  and  responsibilities  for  each  service  contract  (GAO 
Report  05-274,  2005). 

GAO  also  recognized  the  need  for  DoD  to  revise  current  PBS  guidance  on  the 
proper  use  of  other  agencies’  contracts,  to  include  guidance  on  conducting  surveillance  of 
services  that  are  procured  from  other  agencies.  Finally,  GAO  directed  that  the  Anny 
assign  surveillance  personnel  to  conduct  surveillance  of  ongoing  Contract  Advisory  and 
Assistance  Services  as  a  means  of  ensuring  that  service  contracts  are  appropriately 
identified  and  monitored.  This  should  ensure  that  the  Army  is  procuring  services  that  are 
valid  and  required  (GAO  Report  05-274,  2005). 

With  GAO  report  02-1049  titled,  “Guidance  Needed  for  Using  Performance 
Based  Service  Contracting”,  dated  September  2002,  GAO  has  raised  the  concern  that 
agencies  did  not  have  a  clear  understanding  of  PBSAs;  therefore,  agencies  were  not 
taking  full  advantages  of  these  types  of  contractual  instruments.  In  this  report,  GAO  sited 
that  the  Office  of  Federal  Procurement  Policy  (OFPP)  was  aware  of  this  problem  and  was 
working  on  developing  new  guidance  on  the  use  of  perfonnance-based  contracting  (GAO 
Report  02-1049,  2002). 

Consequently,  this  report  recommended  that  OFPP  provide  a  revision  to  their 
PBS  A  guidance  to  ensure  that  PBSAs  were  being  used  appropriately  and  that  proper 
government  oversight  was  being  used  (GAO  Report  02-1049,  2002). 

K.  COMMERCIAL  BEST  PRACTICES 

1.  Metrics 

In  his  article,  “Performance-Based  Service  Metrics  in  IT,”  Peter  S.  Adam  states 
that  for  proper  surveillance  of  IT  PBSCs,  metrics  are  the  key  (Adam,  2003). 

It  is  critical  for  perfonnance  standards  to  be  directly  linked  to  the  desired 
outcomes.  In  addition,  the  performance  standards  have  to  be  measurable,  relevant, 
controllable,  and  achievable  (Adam,  2003). 
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In  order  to  be  measurable,  standards  must  be  quantifiable  and  objective.  In  the 
case  of  IT  PBS  As,  the  measurements  must  be  subjective,  so  a  customer  satisfaction 
survey  may  be  the  best  way  to  measure/assess  contract  performance  (Adam,  2003). 

To  be  achievable,  only  a  few  meaningful  metrics  should  be  used,  and  they  must 
be  set  at  acceptable  service  levels.  Five  to  eight  metrics  are  considered  adequate  and 
must  be  clearly  communicated  to  all  stakeholders  (Adam,  2003). 

Relevant  metrics  must  be  limited  to  only  those  metrics  that  are  truly  important  to 
program  objectives.  When  developing  metrics,  you  must  also  be  sensitive  to  the  cost 
associated  with  measuring  each  objective  to  ensure  the  cost  associated  with  measuring 
the  objective  does  not  exceed  the  value  of  the  measurement.  Expensive  metrics  should 
be  limited  to  only  those  requirements  that  are  mission-  critical  (Adam,  2003). 

Assigned  metrics  must  be  controllable  by  the  contractor.  If  there  are  too  many 
variables  associated  with  the  metrics,  the  contractor  may  not  be  responsible  for  all  the 
variables  that  affect  the  metric  (Adam,  2003). 

2.  Proper  Alignment 

Aligning  the  performance  metrics  with  end  user  objectives  and  customer  needs 
can  be  very  difficult.  Metrics  can  be  either  position  based  or  component  based  (Adam, 
2003). 

Proper  alignment  of  metrics  can  be  very  difficult  as  the  end-users,  contractors, 
and  the  government  may  have  different  perspectives  for  the  performance  of  IT  contracts 
(Adam,  2003). 

What  the  end-users  want  can  be  expressed  as  availability,  performance,  and 
affordability.  Availability  may  be  computer  systems  available  for  the  customer’s  use 
when  and  where  they  are  needed.  Whereas  performance  may  require  computer  systems  to 
work  as  expected  without  interruptions  and  slowdowns  (Adam,  2003). 

Finally,  you  must  determine  if  the  proposed  metrics  are  affordable  and  how  they 
affect  the  “complete”  cost  of  the  system.  When  negotiating  the  contract,  the  parties 
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involved  must  consider  trade-offs  between  price  and  perfonnance,  resulting  in  a  contract 
that  meets  the  end  user’s  need  at  an  affordable  price  (Adam,  2003). 

3.  Customer  Need/Contract  Specifications 

Alignment  of  customer  needs  and  contract  specifications  are  a  must.  To  ensure 
proper  alignment,  it  can  be  beneficial  to  have  the  contractor  propose,  or  at  least  contribute 
to,  the  metrics  and  the  quality  assurance  plan  (QAP).  This  allows  contractors  the 
opportunity  to  develop  their  own  solutions  to  the  requirements  of  the  acquisition.  If  the 
contracting  office  develops  the  metrics  and  QAP  on  their  own,  they  could  be  limiting 
what  the  contractors  are  allowed  to  propose.  The  QAP  must  reflect  reform-mandated 
changes  in  quality  control  and  assurance  of  contracting  practices  (Adam,  2003). 

With  the  Government  moving  away  from  selecting  the  lowest  cost,  technically 
acceptable  proposal,  and  moving  towards  selecting  proposals  based  on  best  value 
techniques,  more  emphasis  is  placed  on  quality  management  and  past-perfonnance 
evaluation,  which  requires  totally  different  incentives  (Adam,  2003). 

4.  The  Quality  Approach 

In  accordance  with  The  Air  Force  Instruction  63-124  (April  1,  1999),  when 
utilizing  the  quality  approach,  contracts  are  tailored  to  manage  risk  and  costs;  best  value 
source  selection  criteria  is  used  that  reduces  oversight  and  ensures  that  only  services 
requested  are  received  and  paid  for;  relies  on  customer  feedback  to  validate  contract 
conformance  or  nonconformance;  allows  flexibility  in  oversight  to  ensure  consistency, 
even  under  changing  circumstances;  allows  contractors,  as  part  of  their  quality  assurance 
system,  to  report  and  perform  on  surveillance  of  services;  and  confirms  surveillance 
results  with  oversight  provided  by  the  Government  Quality  Assurance  (QA)  or  a  third 
party  audit  (Adam,  2003). 

The  PBSA  team  must  realize  that  contract  award  does  not  mean  that  performance 
metrics  are  set  for  the  life  of  the  contract;  they  must  continually  be  reviewed  and  revised. 
Therefore,  the  contract  must  contain  procedures  for  changing  metrics  and  measures  as 
situations  arise  (Adam,  2003). 
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5. 


Incentives 


Incentives  are  vital  when  acquiring  PBSAs,  as  well  as  other  types  of  contracts. 
Contract  incentives  can  be  monetary  or  non-monetary,  positive  or  negative.  The  goal  of 
an  incentive  is  to  motivate  and  encourage  the  contractor  to  provide  the  best  quality 
performance  (Adams  2003). 

The  choice  of  the  type  of  contract  utilized  is  a  non-monetary  incentive  and,  when 
utilized,  can  be  an  incentive  in  an  IT  PBSA.  A  contractor’s  performance  can  be  tied  to 
delivery  and  performance  incentives  using  speed  or  responsiveness  as  an  indicator  of 
meeting  the  metric.  Incentives  can  also  be  based  on  meeting  performance  standard 
targets,  not  just  meeting  minimum  contract  requirements  (Adam,  2003). 

Extension  or  reduction  of  a  contract  for  good  or  sub-  par  performance  can  also  be 
used  as  an  incentive  in  the  contract.  If  the  contractor  fails  to  perform  as  required,  the 
contract  ends  at  the  end  of  the  base  period  (Adam,  2003). 

The  contract  must  clearly  state  the  consequences  of  the  levels  of  performance  in 
terms  such  as  minimum,  maximum,  and  estimated,  and  the  office  responsible  for 
administering  the  contract  must  be  prepared  to  provide  the  appropriate  consequences 
(Adam,  2003). 

The  incentive  structure  must  be  geared  to  the  acquisition,  the  market  place,  and 
the  objectives  to  be  achieved;  there  is  no  “one-size-fits  all”  solution  (Adam,  2003). 

Performance  incentives  can  be  negotiated  and  should  recognize  the  private 
sector’s  most  important  motivator  -  profit.  The  contract  should  be  developed  so  that  the 
Government,  as  well  as  the  contractor,  can  benefit  from  economies  of  scale,  efficiencies, 
and  new  innovations.  Utilizing  correct  incentives  and  ensuring  the  customer’s  needs  are 
met  will  minimize  risks  and  ensure  effective  performance  (Adam,  2003). 

Cost  based  incentives  are  performance  incentives  that  are  designed  to  tie  profit  or 
fee  to  the  results  achieved  by  the  contractor  in  relation  to  cost  based  targets.  An  example 
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for  an  IT  PBSA  would  be  system  availability  rates.  No  matter  which  type  of  measure 
you  choose,  the  perfonnance  measure  must  be  quantified  and  within  a  reasonable  range 
(low-target-high)  (Adam,  2003). 

Award-Fee  contract  arrangements  use  evaluation  factors  provided  in  the  award- 
fee  plan  to  subjectively  assess  a  contractor’s  performance  for  a  specific  evaluation  period. 
Award  fee  pools  are  established  at  the  beginning  of  the  evaluation  period  and  contractors 
are  allowed  to  earn  a  portion  of  an  award  fee  pool  (or  if  performance  allows,  all  of  the 
pool)  (Adam,  2003). 

Award-tenn  contract  arrangements  provide  additional  periods  of  performance  in 
lieu  of  monetary  consideration  for  quality  performance  and  are  a  fairly  new  concept.  The 
contractor’s  incentive  is  tied  to  their  performance  and  if  perfonnance  is  acceptable  then 
the  contractor  gains  an  additional  perfonnance  period.  For  perfonnance  that  is 
substandard,  the  performance  period  can  be  shortened.  These  contracts  are  different  from 
contract  options  as  the  award  term  contracts  are  based  on  formal  evaluation  processes, 
instead  of  regulatory  procedures  as  required  with  priced  options.  In  other  words,  the 
Government  extending  the  contract  tenn  up  to  10  years  can  be  a  reward  good 
performance  (Adam,  2003). 

Schedule  Incentives  -  this  type  of  incentive  focuses  on  the  contractor  meeting  or 
exceeding  the  delivery  schedule.  These  incentives  can  be  stated  using  calendar  days  or 
months,  meeting  or  exceeding  contract  milestones,  or  meeting  urgent  contract 
requirements  (Adam,  2003). 

A  contractor’s  past  performance  rating  can  affect  their  ability  to  continue  to 
perform  past  the  base  year  and  can  affect  their  ability  to  obtain  other  government 
contracts  (Adam,  2003). 

In  the  article  “Making  Performance-Based  Contracting  (PBC)  Perform:  What  the 
Federal  Government  Can  Learn  from  State  and  Local  Governments”,  the  author, 
Lawrence  Martin,  discusses  how  the  Federal  Government  can  learn  from  State  and  Local 
Governments  in  the  way  they  approach  PBCs.  The  article  offers  the  following  lessons 
learned  from  experiences  in  state  and  local  governments  and  provides  recommendations 
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to  the  Office  of  Federal  Procurement  Policy  (OFPP).  Five  lessons  learned  and  four 
recommendations  were  selected  from  this  article  and  they  are  explored  below  (Martin, 
2002). 

6.  Lessons  Learned 

a.  PBC  at  the  state  and  local  government  levels  defines 
“ performance  ”  as  consisting  of  outputs,  quality,  outcomes,  or  any  combination  thereof 
(Martin,  2002). 

Confusion  appears  to  exist  at  the  federal  level  in  terms  of  what  constitutes 
“performance”  in  PBC.  “Performance”  in  state  and  local  government  PBCs  mean 
outputs,  quality,  outcomes,  and  various  combinations.  This  tripartite  conceptualization  of 
performance:  (1)  provides  greater  clarity  as  to  the  purpose  of  PBC,  (2)  gives  state  and 
local  governments  more  options  in  structuring  PBCs,  and  (3)  aligns  more  closely  the 
concept  of  performance  in  PBC  with  the  concept  of  performance  contained  in  the  service 
efforts  and  accomplishments  reporting  initiative  of  the  Governmental  Accounting 
Standards  Board  (Martin,  2002). 

b.  PBC  at  the  state  and  local  government  levels  involves  varying 
degrees  of  being  performance  based  (Martin,  2002). 

A  goal  of  PBC  is  to  make  less  use  of  design  specifications  (input  and 
process)  and  more  use  of  performance  specifications  (outputs,  quality,  and  outcomes). 
The  experience  of  state  and  local  governments  demonstrates  that  service  contract 
specifications  can  be  conceptualized  on  a  continuum  (inputs  — >  process  — »  outputs  — > 
quality  — >  outcomes)  from  “non-PBC”  to  “full-PBC”.  At  the  non-PBC  end  of  the 
continuum  are  service  contracts  that  make  exclusive  use  of  design  specifications;  at  the 
full-PBC  end  of  the  range  are  service  contracts  that  make  exclusive  use  of  outcome 
performance  specifications.  Conceptualizing  PBC  on  a  continuum  allows  for  an 
incremental  and  developmental  approach  to  be  taken  with  certain  services  and  with 
certain  contractors  (Martin,  2002). 

c.  PBC  at  the  state  and  local  government  levels  challenges  the 
notion  that  there  is  one  best  way  to  do  performance-  based  contracting  (Martin,  2002). 
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The  state  of  the  art  in  PBC  today  is  insufficiently  well  developed  to  make 
any  claims  about  how  best  to  implement  this  new  form  of  service  contracting.  What  can 
be  said  is  that  state  and  local  governments  utilize  a  variety  of  different  approaches  in  the 
implementation  of  PBC.  The  common  denominator  in  these  various  approaches  is  the 
conscious  attempt  on  the  part  of  state  and  local  governments  to  change  the  behaviors  of 
the  contractors  to  focus  more  on  performance  (Martin,  2002). 

d.  PBC  at  the  state  and  local  government  levels  include  share-in¬ 
shavings  contracting,  revenue  enhancement  contracting,  and  milestone  contracting 
(Martin,  2002). 

The  experience  of  state  and  local  governments  clearly  demonstrates  that 
share-in-savings  contracting,  revenue  enhancement  contracting,  and  milestone 
contracting  can  affect  the  behavior  of  contractors  to  focus  more  on  perfonnance.  Thus, 
these  approaches  warrant  being  called  PBC.  In  the  case  of  share-in-savings  and  revenue 
enhancement  contracting,  contractor  behavior  is  changed  to  focus  on  accomplishment  of 
certain  processes  and  outputs  that  lead  in  turn  to  the  accomplishment  of  certain  desired 
outcomes  (reduced  service  delivery  costs  and  increased  revenues).  In  the  case  of 
milestone  contracting,  contractor  behavior  is  changed  to  focus  more  on  performance 
because  output,  quality,  and  outcome  performance  requirements,  as  well  as  incentives 
and  penalties,  are  automatically  built  into  the  contract  (Martin,  2002). 

e.  The  step-up/step-down  method  is  a  useful  approach  to 
structuring  incentives  and  penalties  (Martin,  2002). 

Structuring  incentives  and  penalties  to  step-up  and  step-down  from  the 
performance  standard  or  acceptable  quality  level  makes  it  clear  to  contractors  the 
implications  of  acceptable  performance.  Step-up/step-down  incentives  and  penalties  tend 
to  keep  the  contractor  focused  on  the  performance  standards.  Additionally,  the  contractor 
also  has  positive  and  negative  motivations  to  achieve  the  performance  standards.  When 
incentives  and  penalties  step-up/step-down  in  similar  fashion,  and  in  similar  quantities, 
an  appeal  is  also  made  to  fairness  (Martin  2002). 
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7. 


Recommendations 


a.  Operationally  define  “ performance ”  to  include  outputs,  quality, 
outcomes,  or  any  combination  thereof  (Martin,  2002). 

An  operational  definition  of  “performance”  will  resolve  the  current 
conflicting  and  contradictory  operational  definitions  of  PBC  provided  by  OFPP,  which 
views  PBC  as  contracting  for  outputs  and  quality;  the  DoD,  which  views  PBC  as 
contracting  outcomes;  and  the  FAR,  which  views  PBC  as  outcomes  and  quality  (Martin, 
2002). By  providing  an  operational  definition  of  “performance”,  this  will  ensure  that 
Federal  PBC  takes  a  comprehensive  view  of  performance  to  include  considerations  of 
outputs,  quality,  and  outcomes  (Martin,  2002). 

b.  Recognize  that  varying  degrees  of  performance  based  can  exist  in 
an  acquisition  (Martin,  2002). 

This  will  recognize  PBC  as  existing  on  a  range  from  “non-PBC”  to  “full- 
PBC”  and  will  allow  for  an  incremental  and  developmental  approach  to  be  taken  with 
some  services  and  with  some  contractors.  The  current  Federal  perspective  is,  a  service 
contract  is  either  perfonnance-based  or  not  depending  on  the  presence  or  absence  of  four 
essential  elements:  performance  requirements,  performance  standards  or  acceptable 
quality  levels,  a  quality  assurance  or  monitoring  plan,  and  positive  and  negative 
incentives,  if  the  service  is  mission  critical  or  involves  a  large  expenditure  of  public  funds 
(Martin,  2002). 

The  “all  or  nothing”  Federal  perspective  may  result  in  federal  departments 
and  agencies  choosing  not  to  attempt  a  PBC  with  some  services  and  with  some  contractors 
(Martin,  2002). 

c.  Include  share-in-savings,  revenue  enhancement,  and  milestone 
contracting  as  recognized  optional  forms  of  PBC  (Martin,  2002). 

This  will  recognize  the  validity  of  existing  practices.  Share-in-savings, 
revenue  enhancement,  and  milestone  contracting  all  seek  to  change  the  behavior  of 
contractors  to  focus  more  on  perfonnance  (outputs,  quality,  and  outcomes).  As  such, 
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they  constitute  optional  forms  of  PBC  and  should  be  so  recognized  and  will  help  to  begin 
the  process  of  identifying  and  classifying  various  approaches  to  PBC  (Martin,  2002). 


d.  Promote  the  use  of  the  step-up/step-down  method  for  structuring 
incentives  and  penalties  in  PBCs  (Martin,  2002). 

This  recommendation  will  provide  the  Federal  Government  a  simple  and 
easy  way  for  structuring  incentives  and  penalties.  By  structuring  incentives  and 
penalties  to  step-up/step-down  from  performance  standards  or  acceptable  quality  levels; 
it  will  make  it  clear  to  contractors  the  implementations  of  acceptable  and  unacceptable 
performance.  When  incentives  and  penalties  are  developed  in  this  fashion,  an  appeal  is 
made  to  fairness  (Martin,  2002). 

In  the  article  “A  Vision  of  the  Government  as  a  World  Class  Buyer:  Major 
Procurement  Issues  for  the  Coming  Decade”,  the  authors,  Professor  Jacques  S.  Gansler 
and  Roger  C.  Lipitz,  discuss  the  challenges  facing  the  government  regarding  the  effective 
and  efficient  procurement  of  goods  and  services.  A  challenge  they  discuss  in  this  article 
looks  at  changing  the  current  acquisition  process  (Gansler,  2002). 

The  government  must  learn  to  use  incentives,  instead  of  regulations  as  a 
way  to  create  higher  performance  from  their  contractors  and  to  be  able  to  lower  overall 
costs.  If  the  contractors  are  rewarded  for  improving  their  performance  and  lowering  their 
costs,  they  will  make  every  effort  to  make  this  happen.  Incentives  provided  to  the 
contractors  can  be  in  the  form  of  additional  business  or  profit.  The  government  should 
think  in  terms  of  “sharing  the  benefits”  with  the  contractor  to  provide  an  additional 
incentive  for  improving  performance  and  lowering  costs.  In  addition,  you  must  have 
penalties  in  place  in  the  event  the  contractor  does  not  improve  his  perfonnance  and/or 
lower  his  costs  on  the  contract.  The  contractor  should  never  be  given  additional  business 
when  they  repeatedly  do  not  meet  performance  or  cost  measures.  In  the  commercial 
world,  “past  perfonnance”  weighs  heavily  in  the  selection  of  sources,  and  this  should  be 
used  in  government  procurements  as  well  (Gansler,  2002). Significant  improvement  in  the 
government’s  way  of  business  in  determining  their  needs  will  require  a  significant  shift 
from  buying  goods  to  buying  sophisticated  services.  When  buying  sophisticated  services, 
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the  procurements  must  be  “performance  based”.  This  requires  the  government  to  simply 
tell  the  supplier  what  is  needed  instead  of  thinking  of  it  as  buying  labor  hours  at  the 
lowest  rate  possible.  If  the  govermnent  wants  a  high  quality  service,  that  is  what  they 
must  ask  for  and  let  the  suppliers  decide  how  to  many  people  it  will  take  and  what  quality 
of  people  will  be  required  to  do  the  job.  The  government  must  then  learn  how  to  manage 
and  oversee  the  contractors.  Just  because  the  contract  was  given  to  a  private  firm,  does 
not  remove  the  government  from  the  responsibility  of  managing  and  overseeing  the 
contract  (Gansler,  2002). 

This  shift  will  require  considerable  training  for  the  government 
workforce  in  learning  how  to  buy  and  manage  these  sophisticated  services  and  how  to 
create  appropriate  incentives  to  encourage  the  contractors  to  improve  the  quality  of  their 
services  and  to  lower  costs  (Gansler,  2002). 

L.  SUMMARY 

In  conclusion,  this  chapter  provides  the  literary  review  of  PBC,  PBSC,  and  PBSA. 
Additionally,  the  chapter  reviewed  the  elements  required  in  any  PBSA.  In  performing 
the  literary  review,  the  research  project  reviewed  past  and  current  legislative  guidance 
and  policies  on  PBC  and  PBSA,  as  well  as  reviewing  commercial  Best  Practices. 

Chapter  III  will  include  survey  and  interview  questions  and  their  results.  Chapter 
IV  will  compare  these  findings  with  the  expected  results  gathered  during  the  Literary 
Review  in  Chapter  II.  Chapter  V  will  provide  recommendations,  conclusions  and  areas 
for  further  research. 
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III.  SURVEY  AND  INTERVIEW  RESULTS 


A.  INTRODUCTION 

In  order  to  gauge  the  effectiveness  of  existing  IT  PBS  As  and  to  detennine  if 
performance  measurements,  metrics,  and  incentives  are  working  effectively  in 
monitoring  these  contracts,  surveys  and  interviews  were  conducted. 

B.  PURPOSE  OF  SURVEY/INTERVIEWS 

The  purpose  of  the  survey  is  to  compile  cross-comparative  data  regarding  IT 
PBSAs  across  Government  facilities.  A  summary  of  findings  will  be  developed  after 
compilation  of  the  data.  This  survey  is  by  no  means  a  scientific  survey,  does  not 
represent  a  statistical  sampling,  and  the  results  are  not  to  be  determined  representative  of 
results  across  the  Department  of  Army.  The  intention  of  this  survey  was  to  get  a  feel  of 
what  procurement  contracting  staff,  government  technical  staff,  and  contractors  are 
experiencing  with  their  PBSAs  and  to  compare  it  with  the  experiences  of  the  authors. 

C.  DESCRIPTION  OF  THE  PROGRAMS 

The  program(s)  perfonned  by  the  interviewees  and  survey  personnel  included  a 
range  of  IT  services.  The  contractor  personnel  who  were  interviewed  and/or  surveyed 
perform  IT  services  for  several  installations.  A  sample  of  the  IT  services  provided  a 
range  from  Project  Management  support,  help  desk  services,  electronic  mail  support 
services,  blackberry  and/or  telephone  support,  to  end  user  support.  Additionally,  these 
individuals  also  assist  the  Government’s  Most  Efficient  Organization  (MEO)  at  Sierra 
Army  Depot  with  their  reporting  requirements.  In  essence,  they  provide  an  outside  audit 
of  the  Government  contract  for  IT  services. 

The  Government  personnel  who  were  interviewed  and/or  surveyed  provide 
surveillance  and  management  to  include  the  Army’s  vision  of  a  single  Director  for 
Information  at  each  site  and/or  installation.  Currently  at  Rock  Island  Arsenal,  there  are  at 
least  three  Directors  for  Information  Management  who  provide  a  portion  of  the  total 
information  services  for  Rock  Island  Arsenal. 
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D.  SELECTION  OF  RESPONDENTS 

To  ensure  all  members  of  the  PBSA  team  were  allowed  an  opportunity  to  provide 
input,  selected  surveys  were  sent  to  Government  Product  Managers,  Government  IT 
Technical  Staff,  Warranted  Contracting  Officers,  Directors  of  IT  Management  (DOIMs), 
TACOM  contractors,  and  other  Government  procurement  staff.  Selected  interviews  were 
conducted  with  the  President  of  Vista  Corporation,  a  TACOM  contractor,  and  the 
Director  of  IT  for  Rock  Island  Arsenal.  Ten  surveys  were  distributed  and  six  were 
completed  and  returned. 

E.  SURVEY  QUESTIONS 

What  is  your  role  in  IT  contract  services? 

Government  Procurement  Staff 

Government  Technical  Staff 

Contractor 

Question  1: 

Part  A:  What  type  of  contract  are  you  currently  under  for  IT  Services? 

Firm  Fixed  Price 

Cost  Plus 

Time  and  Materials 

Combination  of  Any  of  the  Above 

Other 

Part  B:  What  is  your  opinion  regarding  effectiveness  of  this  contract?  (Were 
desired  outcomes  accomplished?) 

Part  C:  Do  you  feel  this  was  the  appropriate  type  of  contract?  Does  it  meet  your 
anticipated  objectives  and  outcomes? 
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Question  2: 

Part  A:  What  performance  metrics  were  contained  in  the  solicitation/contract  to 
monitor  performance  (cost/schedule/performance  metrics)? 

Part  B:  Were  you  involved  in  the  formulation  of  these  metrics? 

Question  3: 

Part  A:  How  often  do  you  meet  (informally  or  formally)  with  the 

Government/Contractor  team  to  discuss  performance  issues? 

Daily 

Twice  Weekly 

Weekly 

Twice  Monthly 

Monthly 

Every  Other  Month 

Quarterly 

Part  B:  Do  you  feel  that  these  meetings  are  productive? 

Part  C:  What  do  you  believe  is  the  primary  contributing  factor  to  the  productivity 
or  non-productivity  of  the  meetings? 

Part  D:  If  you  had  the  ability  to  make  one  change  to  the  meetings  that  would 
yield  a  better  outcome  to  the  meetings,  what  would  that  change  be? 

Question  4: 

Part  A:  Are  there  incentives  included  in  the  contract,  if  so,  what  are  they? 

Part  B:  Do  you  consider  the  incentives  included  in  the  contract  positive  or 
negative  and  why? 

Part  C:  If  incentives  are  offered,  how  would  you  change  them  to  make  them  more 
effective? 
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Question  5: 

Part  A:  Is  there  a  direct  correlation  between  the  perfonnance  metrics  and  the 
desired  outcomes? 

Part  B:  Is  there  a  direct  connection  between  the  perfonnance  metrics  and  the 
performance  incentives? 

Part  C:  Why  would  you  add,  change,  or  not  change  any  of  the  current 
performance  metrics? 

Question  6: 

Are  the  policies  and  procedures  applicable  and  clear  for  development  of  IT 
PBS  As  SOW,  PWS,  and  SOOs? 

Question  7: 

In  your  opinion,  what  is  the  primary  and  most  important  change  that  needs  to  be 
implemented  to  improve  the  Army’s  IT  PBSAs? 

A  copy  of  the  survey  form  is  at  Exhibit  1 . 

F.  RESULTS  OF  SURVEYS 

The  survey  results  are  as  follows: 

What  is  your  role  in  IT  contract  services? 

Government  Procurement  Staff 

Government  Technical  Staff 

Contractor 

One  respondent  was  a  member  of  the  government  procurement  staff,  four  were 
government  members  of  the  technical  staff,  and  one  respondent  was  a  TACOM 
contractor. 

Question  1: 

Part  A:  What  type  of  contract  are  you  currently  under  forIT  Services? 
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Firm  Fixed  Price 


Cost  Plus 

Time  and  Materials 

Combination  of  Any  of  the  Above 

Other 

Of  the  six  respondents,  the  TACOM  contractor  and  two  members  of  the 
government  technical  staff  stated  their  IT  service  contracts  were  firm  fixed  price.  The 
other  three  respondents,  all  members  of  the  government  technical  staff,  stated  their  IT 
service  contracts  were  a  combination  of  firm  fixed  price,  cost  plus,  and/or  time  and 
materials. 

Part  B:  What  is  your  opinion  regarding  effectiveness  of  this  contract?  (Were 
desired  outcomes  accomplished?) 

When  asked  their  opinion  regarding  the  effectiveness  of  their  firm  fixed  price 
contracts,  the  TACOM  contractor  and  two  government  technical  staff  members  agreed 
that  firm  fixed  price  contracts  were  not  the  best  solution  or  the  most  effective  way  to 
manage  an  IT  service  contract,  especially  with  regards  to  contractor  perfonnance. 

The  remaining  three  respondents,  all  government  technical  staff  members 
working  with  combination  contracts,  felt  these  contract  types  were  very  effective.  One 
respondent  stated  regardless  of  the  type  of  contract,  the  management  of  cost,  schedule, 
and  performance  is  influenced  by  the  education,  training,  and  experience  of  the  project 
managers,  Contracting  Officer’s  Representatives  (CORs),  and  perfonnance  certifiers. 
This  respondent  stated  from  her  experience,  this  is  typically  the  largest  influencing  factor 
related  to  the  effectiveness  of  the  contract,  not  the  contract  type. 

Part  C:  Do  you  feel  this  was  the  appropriate  type  of  contract?  Does  it  meet 
you  anticipated  objectives  and  outcomes? 

When  asked  if  they  felt  the  contract  type  was  appropriate,  the  respondents,  the 

TACOM  contractor  and  two  members  of  the  government  technical  staff  who  had  used 

firm  fixed  price  contracts,  felt  this  type  of  contract  did  not  provide  an  incentive  for 
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contractors  to  continue  providing  better  solutions  to  problems  and  they  also  did  not  allow 
for  incorporating  changes  in  technology  and  to  anticipate  future  workload  requirements. 
In  addition,  the  respondents  felt  firm  fixed  price  contracts  were  a  challenge  to  administer 
and  to  perform  as  well. 

The  remaining  three  members  of  the  government  technical  staff  felt  their 
combination  contracts  allowed  them  flexibility  to  work  with  the  contractor  in  the  ever- 
changing  IT  arena. 

Question  2: 

Part  A:  What  performance  metrics  were  contained  in  the 

solicitation/contract  to  monitor  performance  (cost/schedule/performance  metrics)? 

The  respondents  were  asked  to  describe  the  perfonnance  metrics  contained  in 
their  solicitation/contract  to  monitor  performance. 

None  of  the  respondents  actually  specified  the  performance  metrics  that  were 
included  in  their  solicitation/contract.  Five  of  the  respondents,  the  TACOM  contractor, 
three  members  of  the  government  technical  staff,  and  one  government  procurement  staff 
member  stated  performance  metrics  contained  in  their  contracts  primarily  focused  on 
schedule  and  performance;  these  included  successful  implementation  of  applications, 
reporting  requirements,  a  detailed  perfonnance  requirements  summary,  and  remedies  for 
sub-standard  perfonnance.  The  remaining  respondent,  a  member  of  the  government 
technical  staff,  stated  their  contract  contained  varied  metrics  and  QASP  requirements 
which  ranged  from  successful  implementation  of  applications  to  reporting  requirements. 

Part  B:  Were  you  involved  in  the  formulation  of  these  metrics? 

In  order  to  detennine  who  is  being  included  in  the  development/formulation  of  the 
metrics  placed  in  TACOM  IT  PBSAs,  the  respondents  were  asked  if  they  were  involved 
in  the  formulation  of  the  metrics  in  their  contracts.  Two  individuals  from  the  government 
technical  staff  and  the  one  individual  who  is  part  of  the  government  procurement  staff 
stated  they  were  involved  in  the  formulation  of  the  contract  metrics.  The  other  two 
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individuals  who  are  part  of  the  government  technical  staff  and  the  TACOM  contractor 
stated  they  were  not  part  of  the  fonnulation  of  the  metrics  for  their  contract. 

Question  3: 

Part  A:  How  often  do  you  meet  (informally  or  formally)  with  the 
Government  and/or  the  Contractor  team  to  discuss  performance  issues? 

Daily 

Twice  Weekly 

Weekly 

Twice  Monthly 

Monthly 

Every  Other  Month 

Quarterly 

In  an  effort  to  determine  the  level  of  surveillance  being  done  on  IT  PBS  As, 
respondents  were  asked  how  often  the  government/contractor  team  meets  to  discuss 
performance  issues. 

Two  respondents,  both  government  technical  staff  members,  stated  their  teams 
have  daily  meetings;  a  procurement  staff  member  stated  their  team  meets  twice  a  week; 
one  government  technical  staff  member  stated  their  team  meets  weekly,  monthly,  and 
quarterly;  another  government  technical  staff  member  stated  their  team  meets  monthly; 
and  finally,  the  TACOM  contractor  stated  he  meets  with  the  government  team  every 
other  month. 

Part  B:  Do  you  feel  that  these  meetings  are  productive? 

When  asked  if  their  meetings  were  productive,  the  TACOM  contractor,  the 
government  procurement  staff  member,  and  three  government  technical  staff  members 
stated  their  meetings  were  productive,  while  one  government  technical  staff  member 
stated  their  meetings  were  not  productive. 
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Part  C:  What  do  you  believe  is  the  primary  contributing  factor  to  the 
productivity  or  non-productivity  of  the  meetings? 

In  order  to  determine  why  meetings  were  or  were  not  productive,  the  respondents 
were  asked  what  they  believed  was  the  primary  contributing  factor  that  made  the  meeting 
productive  or  nonproductive. 

One  member  of  the  government  technical  staff  stated  having  an  agenda,  a 
structured  format  for  the  meeting,  and  clear/concise  expectations  made  their  meeting 
productive.  In  addition,  this  respondent  feels  communication  must  be  simple  and  frank 
and  that  talking  around  issues  is  ineffective  in  getting  them  resolved. 

Another  government  technical  staff  member  feels  open  discussions  with  the 
contractor  and  understanding  their  concerns  will  help  the  team  stop  a  problem  before  it 
happens.  Limiting  discussions  to  critical  issues,  repetitive  performance  problems,  and 
high  cost  areas  are  contributing  factors  stated  by  one  government  procurement  staff 
member  and  a  government  technical  staff  member  would  help  make  meetings  more 
productive. 

A  contractor  who  is  unwilling  or  unable  to  perform  is  stated  as  a  contributing 
factor  by  one  of  the  government  technical  staff  members.  Finally,  the  respondent,  who  is 
a  TACOM  contractor,  feels  meetings  are  necessary  due  to  a  difference  in  understanding 
of  issues  by  the  government  and  industry.  He  feels  these  meetings  foster  an 
understanding  of  the  contractor’s  perfonnance  and  help  build  trust  between  the  customer 
and  the  contractor. 

Part  D:  If  you  had  the  ability  to  make  one  change  to  the  meetings  that  would 
yield  a  better  outcome  to  the  meetings,  what  would  that  change  be? 

When  asked  if  you  had  the  ability  to  make  one  change  in  the  meetings  in  order  to 
yield  a  better  outcome,  what  would  that  suggestion  be?  The  following  responses  were 
received. 

Provide  a  clear  and  concise  expectation  with  clear  measurable  outcomes  was  the 
suggestion  from  one  of  the  government  technical  staff  members. 
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Another  government  technical  staff  member  stated  he  would  limit  the  acceptable 
range  of  allowable  excuses  for  failed  perfonnance  to  reasonable  issues  only.  If  either 
party  failed  due  to  their  lack  of  appropriate  application  of  resources,  accept  the  deficiency 
and  move  on. 

All  parties  should  be  less  adversarial  states  another  government  technical  staff 
member. 

The  government  procurement  staff  member  feels  limiting  discussions  to  the  most 
critical  issues  would  increase  productivity  during  their  meetings. 

During  the  meetings,  suggest  an  improvement  to  the  perfonnance/reports  so  they 
can  be  brought  to  the  next  meeting  was  a  suggestion  by  the  TACOM  contractor. 

Question  4: 

Part  A:  Are  there  incentives  included  in  the  contract,  if  so,  what  are  they? 

Part  B:  Do  you  consider  the  incentives  included  in  the  contract  positive  or 
negative  and  why? 

To  determine  what  incentives  are  being  used  in  TACOM  contracts,  each 
respondent  was  asked  to  describe  what  incentives  were  used  in  their  contracts  and  if  the 
incentives  were  positive  or  negative. 

Two  respondents,  one  a  government  technical  staff  member  and  the  other  a 
TACOM  contractor,  stated  there  were  no  incentives  in  their  contract.  Though,  the 
government  technical  staff  member  went  on  to  say  the  government,  in  most  cases,  isn’t 
comfortable  or  familiar  with  trying  to  include  financial  incentives,  at  least  from  his 
experience.  Other  incentives  are  typically  not  used  or  encouraged  -  it  takes  consistent 
management  of  the  contract  to  ensure  performance  and  validation  in  order  to  approve 
incentives,  which  isn’t  always  done  or  available  through  the  government  team. 

Another  respondent,  a  government  technical  staff  member,  stated  there  were  no 
positive  incentives  in  their  contract,  but  there  were  a  significant  amount  of  penalties  for 
non-perfonnance  of  the  contract. 
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Two  respondents,  a  government  procurement  staff  member  and  a  government 
technical  staff  member,  stated  the  incentive  included  in  their  contracts  tied  extension  of 
the  life  of  the  contract  with  exceptional  performance. 

The  final  respondent,  a  government  technical  staff  member,  stated  incentives  in 
their  contract  are  tied  to  perfonnance  and  they  are  positive  incentives  as  the  contractor  is 
compensated  for  completed  work  within  the  quality  standards.  If  the  contractor’s  work  is 
not  in  accordance  with  the  standards,  then  a  deduction  can  be  made. 

Part  C:  If  incentives  are  offered,  how  would  you  change  them  to  make  them 
more  effective? 

When  offered  the  chance  to  change  the  contract  incentives  to  make  them  more 
effective,  only  two  government  technical  staff  members  provided  responses.  One  of  the 
government  technical  staff  members  stated  he  would  like  to  see  some  of  the 
government’s  financial  regulations  and  the  way  the  government  handles  their  money 
changed.  The  other  technical  staff  member  stated  he  would  make  sure  the  government 
and  the  contractor  fully  understand  the  linkage  between  performance,  compensation, 
deductions,  and  how  they  will  be  applied  during  the  course  of  the  contract. 

Question  5: 

Part  A:  Is  there  a  direct  correlation  between  the  performance  metrics  and 
the  desired  outcomes? 

All  respondents  felt  there  is  a  direct  correlation  between  the  performance  metrics, 
the  desired  outcomes,  and  how  desired  outcomes  are  measured  against  the  contract 
provisions.  The  TACOM  contractor  stated  the  perfonnance  metrics  in  their  current 
contract  were  very  ineffective  in  determining  the  performance  of  the  contract.  Without 
the  valuable  input  of  the  industry  representative,  the  chance  of  accurate  measurements 
has  been  skewed. 
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Part  B:  Is  there  a  direct  connection  between  the  performance  metrics  and  the 
performance  incentives? 

When  asked  if  there  is  a  direct  connection  between  the  perfonnance  metrics  and 
the  perfonnance  incentives,  the  government  procurement  staff  member  and  two 
government  technical  staff  members  said  yes  there  was;  the  TACOM  contractor  and  two 
government  technical  staff  members  stated  no  there  was  not;  and  one  government 
technical  staff  member  did  not  answer  this  question. 

Part  C:  Why  would  you  add,  change,  or  not  change  any  of  the  current 
performance  metrics? 

The  respondents  were  asked  why  would  you  change,  not  change,  or  add 
performance  metrics  to  the  current  contract. 

The  government  procurement  staff  member  and  two  government  technical  staff 
members  did  not  answer  this  question;  one  government  technical  staff  member  stated 
every  contract  has  to  be  considered  individually  to  determine  what  needs  to  be  changed 
and  adapted  based  on  lessons  learned;  and  the  TACOM  contractor  stated  current  metrics 
should  be  changed  to  reflect  a  positive  direction  and  to  provide  cost  effectiveness  and 
performance  solutions.  Metrics  should  be  present  as  a  dashboard  for  improvement  and 
not  a  chopping  block  if  they  are  not  met. 

Question  6: 

Are  the  policies  and  procedures  applicable  and  clear  for  development  of  IT 
PBS  As  SOW,  PWS,  and  SOOs? 

This  question  asks  the  respondents  if  current  policy  and  procedures  are  applicable 
and  clear  for  development  of  IT  PBS  As,  SOWs,  PWS,  and  SOOs.  One  government 
technical  representative  answered  the  policies  and  procedures  were  not  applicable  and 
clear  causing  the  government  technical  staff  to  rely  on  the  contracting  officers  for 
assistance. 

Another  government  technical  representative  answered  no,  but  provided  no 
reasons  for  his  answer. 
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The  positive  responses  came  from  the  government  procurement  staff  member, 
two  government  technical  staff  members,  and  the  TACOM  contractor.  Only  the  TACOM 
contractor  provided  comments. 

The  TACOM  contractor  stated  policies  and  procedures  are  applicable,  but  remain 
vague  in  the  development  of  IT  PBSAs,  SOWs,  PWS,  and  SOOs.  He  feels  this  is  good  as 
each  area  can  define  their  own  needs  and  requirements  for  performance  of  IT  services 
and  support.  The  value  of  having  an  outside  industry  expert  in  developing  these  criteria 
is  a  vital  step,  without  a  common  ground  of  understanding;  often  performance  metrics 
will  be  far  from  the  intended  target. 

Question  7: 

In  your  opinion,  what  is  the  primary  and  most  important  change  that  needs 
to  be  implemented  to  improve  the  Army’s  IT  PBSAs? 

Each  respondent  was  asked  “what  is  the  primary  and  most  important  change  that 
needs  to  be  implemented  to  improve  the  Anny’s  IT  PBSAs”.  The  answers  are  as  follows: 

One  government  technical  staff  member  stated  that  less  emphasis  should  be 
placed  on  cost  and  more  emphasis  should  be  placed  on  other  factors  such  as  technical 
merits  of  the  bid  and  the  contractor’s  past  perfonnance.  Past  perfonnance  needs  to  be 
scrutinized. 

When  contracts  are  being  awarded  for  IT  work,  one  government  technical  staff 
member  would  like  them  to  be  routed  through  the  IT  office  to  verify  the  application  and 
software  that  can  be  applied  to  equipment  within  the  government.  Sometimes  unforeseen 
costs/time  occurs  due  to  lack  of  coordination. 

The  government  procurement  staff  member  feels  that  more  samples  and  lessons 
learned  need  to  be  made  available. 

Develop  education  and  consistent  practices,  checklists  for  buyers  to  look  for  as  it 
relates  to  IT  acquisitions,  and  increase  education  for  the  workforce  on  IT  acquisition 
management  is  the  suggestion  of  one  government  technical  staff  member. 
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One  government  technical  staff  member  feels  that  IT  personnel  need  more  than 
classroom  training  to  be  effective  in  development  of  IT  contract  vehicles.  Hands-on 
experience  is  by  far  the  most  beneficial  training  method.  For  the  most  part,  the 
Information  Management  Specialist  is  poorly  equipped  to  absorb  and  understand  the 
fundamentals  of  the  acquisition  community,  let  alone  its  subtleties. 

Finally,  the  TACOM  contractor  feels  that  the  Army  should  continue  to  look  out  to 
the  corporate  world  and  develop  some  of  the  metrics  and  incentives  used  in  IT  PBSAs. 
Dashboards,  milestones,  balance  scorecards,  and  goals  and  controls  are  just  some  of  the 
performance  criteria  currently  being  used  to  measure  and  encourage  performance.  The 
one  thing  all  of  these  programs  have  in  common  is  the  customer  and  the  contractor  come 
together  to  discuss  and  detennine  what  the  measurements  should  be  to  foster  a  win-win 
situation  during  the  contract. 

G.  INTERVIEW  QUESTIONS  AND  RESULTS 

The  results  from  both  Government  and  Contractor  IT  Management  Personnel  are 
as  follows: 

Question  1:  This  question  asked  if  the  type  of  contract  was  appropriate  for 
the  type  of  work  required  by  the  contract. 

Both  Government  and  Contractor  personnel  interviewed  are  currently  using  firm 
fixed  price  contracts  and  felt  that  this  type  of  contract  was  not  appropriate  for  the  work 
required  under  their  contracts.  There  are  areas  where  a  fixed  price  line  item  is 
appropriate,  but  there  are  also  line  items  that  need  more  flexibility  than  a  firm  fixed  price 
contract  allows,  i.e.,  a  line  item  where  the  service  involves  changing  technologies  or 
application  development/maintenance  may  fit  better  as  level  of  effort  line  item. 
Consequently,  the  most  appropriate  contract  type  would  be  one  that  provides  the  most 
flexibility  and  can  accommodate  changing  technology  and  fluctuations  in  budget  and 
workload. 

Question  2:  Did  The  Team  Receive  Any  Training  On  PBSA? 

When  asked  if  the  team  received  training  on  PBS,  the  Government  IT  Manager  at 
TACOM-RI,  stated  that  his  team  did  not  received  training  on  PBS.  This  person  also 
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stated  that  team  training  was  not  emphasized,  nor  was  the  team  involved  in  the 
development  of  the  acquisition  strategy  or  the  actual  solicitation/contract  since  the 
acquisition  was  developed  under  the  A-76  program.  The  secrecy  of  the  A-76  process  did 
not  lend  itself  to  ensuring  that  all  parties  had  a  thorough  understanding  of  what  type  of 
training  and  contractual  requirements  were  needed  to  ensure  that  the  Government  was 
getting  a  fair  comparison  from  both  the  Government  and  private  industry. 

While  the  government  personnel  did  not  receive  training,  both  TACOM  IT 
contractors  stated  that  their  firms  ensure  that  each  program  manager  is  provided  training 
on  PBSA.  In  fact,  one  of  the  TACOM  IT  contractors  stated  that  his  program  managers 
receive  the  same  training  that  is  offered  to  Government  acquisition  personnel.  This 
contractor  feels  that  this  helps  his  company  in  building  a  relationship  with  the 
Government  after  the  contract  is  awarded.  In  his  opinion,  he  feels  that  it  enables  his 
company  to  help  the  Government  further  define  the  desired  outcomes  as  the  contract  is 
performed  and  to  deal  with  the  unintended  consequences  as  they  occur. 

Question  3:  Were  the  desired  outcomes  clearly  stated  in  the  contract? 

Both  the  Government  and  Contractor  interviewees  stated  that  the  desired 

outcomes  were  not  clearly  stated.  One  of  the  TACOM  IT  Contractors  stated  that  he 

would  review  sampling  procedures/requirements  in  the  QASP  and  make  sure  that 

processes  were  in  place  to  insure  adequate  surveillance  of  the  contract.  However,  the 

Government  manager  did  state  that  there  was  no  distinction  between  the  critical  and 

routine  components  of  the  services  required  under  the  contract  and,  in  some  cases;  this 

diluted  the  effectiveness  of  having  the  QASP.  Two  of  the  three  interviewees  stated  that 

the  QASP  is  not  effective  in  achieving  the  desired  outcomes  because  the  Government 

failed  to  properly  identify  the  desired  outcomes  from  the  onset.  As  the  Government 

Manager  stated  in  his  interview,  the  unintended  consequence  of  not  clearly  defining  the 

desired  outcomes  or  not  knowing  what  truly  was  a  desired  outcome  resulted  in  the 

Government  being  forced  to  perform  more  detail  surveillance  than  originally  intended. 

Consequently,  this  was  a  very  costly  consequence  of  outsourcing  for  both  the 

Government  and  the  private  contractor.  The  private  contractor  agreed  that  the 

Government  did  not  clearly  define  their  desired  outcomes.  He  stated  that  the 
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Government  does  not  necessarily  know  how  to  define  their  metrics.  In  his  opinion,  the 
Government  personnel  do  not  know  how  to  measure  perfonnance  using  metrics; 
therefore,  it  is  hard  for  them  to  know  how  or  what  their  desired  outcomes  are.  The 
contractor  substantiated  his  statement  by  stating  that  his  firm  has  been  assisting  the 
Government  Most  Efficient  Organization  (MEO),  at  a  subordinate  command,  with 
refining  their  metrics  and  providing  reporting  data.  In  fact,  this  information  provided 
with  the  contractor’s  assistance  resulted  in  the  Government  organization  requiring 
additional  personnel  to  perfonn  the  required  services. 

In  conclusion,  both  all  three  interviewees  provided  candid  responses  and  stated 
that  PBSA  is  the  correct  tool  to  use  for  IT  PBSAs,  however,  there  are  a  lot  of  lessons 
learned  that  need  to  be  considered  when  the  next  contract  is  developed. 

H.  SUMMARY 

Ten  surveys  were  distributed,  with  six  surveys  completed  and  returned. 
Additionally,  three  interviews  were  conducted  based  on  information  received  from  the 
surveys.  Survey  and  interview  results  have  been  compiled  in  Chapter  III.  Chapter  IV 
compared  the  real  world  findings  with  the  expected  results  gathered  during  the  Literary 
Review  in  Chapter  II. 
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IV.  FINDINGS  AND  RESULTS  (LITERARY  REVIEW  AND 
SURVEY/INTERVIEW  RESULTS 

A.  INTRODUCTION 

This  chapter  will  provide  a  comparison  of  the  findings  discussed  in  Chapter  III 
with  the  expected  results  discussed  in  Chapter  II  of  this  research  project.  As  stated  in 
Chapter  II,  the  literary  review  provided  the  basis  and  rationale  why  PBSAs  are  so 
important.  The  importance  of  any  PBSA  is  to  assist  the  organization(s)  with  perfonnance 
measurements  and  objectives.  Additionally,  the  surveys  and  interviews  included  in 
Chapter  III  indicate  real  world  experiences.  Based  on  these  findings  and  results,  Chapter 
IV  will  provide  recommendations  and  conclusions. 

B.  COMPARISON  OF  FINDINGS  TO  EXPECTED  RESULTS 

In  performing  a  comparison  of  the  information  gathered  in  the  literature  review 
with  the  results  of  the  surveys  and  interviews,  the  research  identified  various  disconnects 
or  deficiencies  in  the  PBSA  process.  These  deficiencies  and  disconnects  were  identified 
in  the  following  areas:  establishing  the  PBSA  team,  defining  the  requirement,  conducting 
market  research,  solicitation  development  (SOW,  PWS,  Measuring  Performance 
Standards,  Metrics,  Incentives  and  Remedies,  Awarding  the  Contract,  and  Contractor 
Surveillance.  The  remainder  of  this  chapter  will  discuss  each  of  these  areas  in  detail. 

1.  Establishing  of  the  PBSA  Team 

The  literary  review,  conducted  in  Chapter  II,  stated  that  establishment  of  the 
acquisition  team  is  essential  to  the  success  of  any  PBSA.  To  stress  the  importance  of  the 
development  of  the  acquisition  team,  the  FAR  defines  the  acquisition  team  in  the  Guiding 
Principles.  This  is  done  to  ensure  participants  are  identified  beginning  with  the  customer 
and  ending  with  the  contractor  who  has  performed  the  required  services.  By  identifying 
the  team  members  in  this  manner,  teamwork,  unity  of  purpose,  open  communication, 
team  members  sharing  the  vision  and  achieving  the  goal  of  the  acquisition  is  encouraged 
(FAR  Part  1.102-3). 
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The  DoD  Directive  5000.2  indicates  that  with  any  acquisition,  the  Integrated 
Product  Team  (IPT)  should  be  comprised  of  technical  personnel,  acquisition  personnel, 
and  contractor  personnel,  depending  on  the  acquisition  (DoDI  5000.2,  2004). 

The  results  of  the  surveys  and  interviews  discovered  that  development  of  the 
acquisition  team  was  not  being  completed  as  required.  One  of  the  Government  IT 
management  personnel  interviewed  stated  that,  in  their  situation,  the  lack  of  participation 
by  the  PBS  A  team  was  possibly  a  result  of  the  solicitation/contract  being  developed 
under  the  A-76  cost  comparison  process,  and  that  the  secrecy  associated  with  this  process 
did  not  allow  the  government  technical  personnel  and  contractors  to  work  as  a  team. 

In  addition,  all  survey  respondents,  both  government  and  contractor  personnel, 
felt  that  to  be  a  more  informed  participant  in  the  PBSA  team,  the  Government  technical 
personnel  need  to  receive  training  in  the  development  of  IT  PBSAs.  Additionally,  one  of 
the  Government  IT  Managers  surveyed  stated  they  felt  that  as  part  of  the  team  they 
should  also  receive  training  in  the  fundamentals  of  acquisition.  Still,  another  survey 
indicated  that  acquisition  personnel  needed  to  have  a  clear  understanding  of  the  expected 
results  in  order  to  assist  the  team  with  the  best  acquisition  strategy. 

2.  Defining  the  Requirement 

To  define  the  requirement,  the  literary  review  states  the  PBSA  team  needs  to 
describe  the  problem  to  be  solved  using  performance  specifications  (outputs,  quality,  and 
outcomes)  instead  of  design  specifications  (inputs  and  processes)  (Martin  2002).  In 
addition,  when  defining  the  requirement,  the  agency  must  develop  performance  goals  and 
objectives,  which  can  be  included  in  the  PWS  or  SOO  (Seven  Steps  to  PBSAs,  2005). 

With  the  exception  of  the  government  procurement  staff  member,  only  two  of  our 
respondents,  both  government  technical  staff  members,  were  involved  in  the  development 
of  the  requirement. 

A  comment  by  one  of  the  respondents  was  that  the  secrecy  of  the  A-76  process 
did  not  lend  itself  to  ensuring  all  parties  had  a  thorough  understanding  of  what  type  of 
contractual  requirements  were  needed  to  ensure  the  government  would  get  a  fair 
comparison  from  both  the  government  and  private  industry. 
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Additionally,  the  literary  infonnation  provided  in  Chapter  II  stated  that  industry 
members  feel  that  the  government’s  inability  to  clearly  define  their  requirement  stems 
from  a  lack  of  training  and  experience  in  PBSAs  (GAO  Report  02-1049,  2002).  This  was 
echoed  by  comments  made  in  both  the  surveys  and  interviews  conducted  for  this  research 
project.  In  fact,  the  TACOM  contractor  interviewed  felt  the  QASPs  are  ineffective  and 
should  not  be  used  in  PBSAs.  Consequently,  the  PWS  and  QASP  are  always  being 
modified  due  to  the  government’s  inability  to  clearly  define  requirements. 

The  remaining  government  respondents  were  not  included  in  defining  the 
requirements  of  the  acquisitions,  even  though  they  would  ultimately  be  responsible  for 
providing  surveillance  assistance  for  the  resulting  contract. 

However,  the  Government  technical  personnel  felt  the  PWS  clearly  defined  the 
requirements  or  at  least  the  desired  outcomes  and  the  subsequent  QASP  provided  the 
contractor  with  enough  information  for  them  to  perform  the  required  services. 
Consequently,  it  is  important  for  the  PBS  A  team  to  have  a  clear  understanding  of  the 
services  required  in  any  PBSA  (Seven  Steps  to  PBSAs,  2005). 

3.  Conducting  Market  Research 

Market  research  is  the  responsibility  of  the  PBSA  team.  They  must  examine  both 
public  and  private  sector  solutions  to  their  stated  requirement.  Even  though  the 
government  has  stated  its  preference  for  firm  fixed  price  contracts  for  IT  PBSAs,  the 
literary  review  provides  that  market  research  is  a  good  time  for  determining  the 
appropriate  contract  type  to  fit  the  stated  requirement,  which  may  be  a  hybrid  contract 
that  contains  both  fixed  price  and  cost  reimbursement  performance  measures  (Seven 
Steps  to  PBSAs,  2005). 

The  surveys  and  interviews  did  not  specifically  ask  if  PBSA  team  members  were 
involved  in  the  market  research  for  their  particular  acquisitions.  We  could  assume  that 
the  teams  that  chose  other  than  firm  fixed  price  contracts  did  some  type  of  market 
research. 

In  discussing  this  issue  with  the  contracting  officers  that  have  worked  with  PBSA 
contracts  at  TACOM-RI,  the  very  first  PBSA  contract  was  a  result  of  an  A-76  study.  For 
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this  acquisition,  the  PBSA  team  was  not  involved  in  the  development  of  the  requirement 
or  in  the  market  research.  Still,  the  research  conducted  for  this  project  did  indicate  that  a 
sources  sought  synopsis  was  issued  for  this  acquisition  as  a  means  of  soliciting  interested 
parties.  As  this  was  the  only  market  research  conducted  and  due  to  the  fact  the  PBSA 
team  was  not  involved,  adequate  market  research  may  not  have  been  conducted,  which 
could  have  lead  to  problems  with  the  development  of  the  SOW,  award  of  the  contract, 
and  contract  surveillance.  In  conclusion,  market  research  is  essential  to  the  success  of  any 
PBSAs  (Seven  Steps  to  PBSAs,  2005). 

4.  Solicitation  Development  -  SOW,  Metrics,  Incentives,  etc. 

The  literary  review  conducted  in  Chapter  II  states  that  the  solicitation  and  the 
required  services  must  be  written  in  support  of  mission-based  performance  goals.  This  is 
further  reiterated  in  Step  2  of  the  Seven  Steps  to  PBSAs,  wherein  it  states  the 
performance  goals  set  the  stage  for  the  development  of  the  solicitation  (Seven  Steps  to 
PBSAs,  2005,  Step  Two).  The  survey  results  indicated  that  the  PBSA  team  did  not 
necessarily  know  what  their  intended  outcome  or  perfonnance  objective  was,  therefore, 
they  were  unable  to  clearly  identify  how  the  mission  goals  related  to  the  solicitation 
development.  One  of  the  reasons  given  for  this  problem  could  have  been  attributed  to  the 
secrecy  of  the  A-76  cost  comparison  process  and/or  the  lack  of  training  and  experience 
on  the  part  of  the  technical  personnel. 

The  literary  review  also  provided  an  example  on  how  an  IT  PBSA  could  be 
structured  to  meet  the  organization’s  mission.  The  example  illustrates  the  need  for  an  IT 
organization  to  provide  a  local  LAN  network  and  infrastructure  that  can  provide  its 
customers  with  the  highest  quality  and  service.  Thus,  the  focus  or  objective  of  the 
acquisition  should  be  on  quality  and  customer  satisfaction.  This  should  enable  the 
organization  to  meet  their  organizational  objectives  of  ensuring  the  organization  “...  has 
a  cost-effective,  efficient,  accessible,  and  reliable  network  infrastructure  with  modern 
workplace  software  and  hardware,  to  promote  productivity  and  meet  business  needs” 
(Seven  Steps  to  PBSAs,  2005).  However,  as  previously  stated,  the  research  surveys  and 
interviews  indicated  that  the  team  and  the  acquisition  did  not  clearly  identify  the  intended 
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results.  As  a  result,  the  contract  did  not  necessarily  meet  the  needs  of  the  organization 
and/or  support  the  performance  objectives  as  intended  by  the  GPRA  and  illustrated  in 
Figure  1  (Seven  Steps  to  PBSAs,  2005). 

Figure  1: 

PERFORMANCE  PLANNING  AND  PERFORMANCE- 

BASED  ACQUISITION 


GPRA  Strategic  Plan 
Requirement 

Related  Annual 
GPRA  Performance 
Plan  Requirement 

Incorporation  in 
Acquisition 

A  comprehensive  mission 
statement  covering  the  major 
functions  and  operations  of  the 
agency. 

No  counterpart. 

Establishes  the  function  to  be 
supported  by  the  technology  as 
central  to  achievement  of  the 
agency’s  mission,  answering  in 
part  the  first  "pesky  question"  - 
and  makes  clear  to  contractor(s) 
the  mission-related  role  of  the 
contract. 

General  goals  and  objectives, 
including  outcome-related 
goals  and  objectives,  for  the 
major  functions  and  operations 
of  the  agency. 

Performance  goals,  expressed 
in  an  objective,  quantifiable, 
and  measurable  form,  that 
define  the  level  of  performance 
to  be  achieved  by  a  program 
activity. 

Enables  the  statement  of  the 
requirement  to  address  desired 
outcomes,  placing  a  degree  of 
performance  burden  on  the 
contractor.  [Note:  For  all  but  major 
acquisitions,  the  GPRA 
performance  goals  may  need  to  be 
expanded  to  detail  the  cost, 
schedule,  and  performance  goals 
required  by  FASA.] 

A  description  of  how  the  goals 
and  objectives  are  to  be 
achieved,  including  a 
description  of  the  operational 
processes,  skills  and 
technology,  and  the  human, 
capital,  information,  and  other 
resources  required  to  meet 
those  goals  and  objectives. 

The  operational  processes, 
skills  and  technology,  and  the 
human,  capital,  information,  or 
other  resources  required  to  meet 
the  performance  goals.  [Note: 
the  description  in  the  annual 
plan  may  be  more  detailed  than 
that  in  the  strategic  plan.] 

May  be  used  to  flesh  out  the 
statement  of  requirements  in  terms 
of  an  approved  (by  agency,  OMB, 
and  Congress)  approach  and 
required  resources. 

A  description  of  the  program 
evaluations  used  in  establishing 
or  revising  general  goals  and 
objectives,  with  a  schedule  for 
future  program  evaluation. 

Performance  indicators  to  be 
used  in  measuring  or  assessing 
the  relevant  outputs,  service 
levels,  and  outcomes  of  each 
program  activity. 

The  basis  for  comparing  actual 
program  results  with  the 
established  performance  goals. 

The  means  to  be  used  to  verify 
and  validate  measured  values. 

Permits  development  of  a 
complementary  contractor 
evaluation  process  (to  the  extent 
that  the  contractor  will  be 
responsible  for  meeting 
performance  goals  and  objectives) 
that  can  feed  into  the  program 
evaluation  process.  [Note:  For  all 
but  major  acquisitions,  the  GPRA 
performance  indicators  may  need 
to  be  expanded  to  detail  how  to 
measure  the  cost,  schedule,  and 
performance  goals  required  by 
FASA.] 

49 


As  illustrated  above,  the  chart  depicts  the  linkage  between  an  organization’s 
strategic  plan,  their  annual  plan,  and  the  PBSA.  Thus,  this  enables  the  organization  to 
link  the  funding  required  to  meet  mission  objectives  to  performance  requirements  and 
desired  outcomes.  Note  that  the  focus  here  is  not  on  the  resources  required,  but  on  what 
outcome  is  required.  With  this  foundation,  when  the  planning  process  is  complete,  an 
agency  should  be  able  to  demonstrate  clearly  how  an  individual  acquisition's  performance 
objectives  will  assist  in  achieving  the  agency's  mission  and  goals  (Seven  Steps  to  PBSA, 
2005). 

a.  PWS,  SOO,  and/or  SOW 

As  the  literary  review  indicated,  once  the  performance  goals  are  identified, 
the  team  should  switch  their  focus  to  the  desired  outcomes  and  objectives  surrounding 
the  acquisition  (KnowNet,  2005).  The  survey  results  and  subsequent  interviews 
indicated  that  the  SOW,  PWS,  and/or  the  SOO  did  not  support  the  desired  outcomes.  As 
the  contractor  personnel  indicated  in  their  interviews,  their  experience  has  shown  that 
with  IT  PBS  As,  the  SOW  is  often  used  as  a  living  document  that  must  be  modified  in 
order  to  get  the  outcomes  desired.  The  problem  with  this  type  of  philosophy  is  that  it  can 
affect  the  integrity  of  the  acquisition,  as  other  contractors  did  not  have  the  same 
opportunity  to  submit  their  proposals  to  the  revised  SOW,  which  actually  provided  the 
desired  outcomes. 

Additionally,  the  government  personnel  surveyed  and  interviewed  also 
stated  the  PWS  did  not  clearly  identify  desired  outcomes.  The  Government  technical 
team  stated  that  they  understood  the  intent  of  the  IT  PBSA  and  felt  that  the  contractor 
also  should  have  clearly  understood  what  was  required  under  this  contract.  However, 
their  vision  could  have  been  clouded  by  the  fact  that  they  were  the  contractor’s 
competitors  and  had  previously  performed  these  tasks  as  government  employees.  As  the 
literary  review  has  shown,  there  is  no  standard  template  to  provide  the  PBSA  team  with 
assistance  in  writing  a  PWS  (Seven  Steps  to  PBSA,  2005).  In  addition,  FAR  guidance 
only  requires  that  the  PWS  describe  the  requirements  in  terms  of  results  rather  than 
processes:  use  measurable  performance  standards  and  QASPs;  provide  for  reduction  of 
fees  or  price;  and  include  performance  incentives  as  appropriate  (Seven  Steps  to  PBSA, 
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2005).  For  government  personnel  who  are  not  familiar  with  PBSAs,  the  FAR  guidance 
does  not  provide  adequate  information  for  development  of  critical  parts  of  the  PBSAs. 
This  will  be  discussed  further  in  this  chapter  and  in  Chapter  V. 

b.  Measurable  Performance  Standards  or  Metrics 

The  literary  review  in  Chapter  II  showed  that  performance  standards  or 
metrics  play  an  important  part  in  any  PBS  A  as  they  are  directly  linked  to  the  desired 
outcome(s)  (Seven  Steps  to  PBSA,  2005).  However,  the  use  of  too  many  metrics  can  be 
costly  both  to  the  government  and  to  the  contractor.  The  more  metrics  contained  in  a 
contract  the  more  technical  personnel  are  required  for  contractor  surveillance. 
Additionally,  the  contract  cost  increases  with  every  metric  imposed  on  the  contractor. 
Consequently,  the  PBSA  team  should  only  include  metrics,  which  are  relevant  to  desired 
outcomes,  as  well  as  being  measurable.  Therefore,  as  a  rule  of  thumb,  PBSAs  should 
only  contain  one  performance  metric  for  each  task  unless  otherwise  required  (KnowNet, 
2005). 

In  comparison,  none  of  the  survey  respondents  indicated  that  they  were 
involved  in  the  development  of  any  metrics  used  on  their  PBSAs.  In  fact,  the  Rock 
Island  Arsenal  DOIM  stated  in  that  some  of  the  requirements,  under  the  PBSA,  were 
written  by  another  organization.  Yet,  the  primary  reason  for  the  use  of  metrics  in  their 
contracts  was  to  focus  on  schedule  and  performance;  this  included  successful 
implementation  of  applications,  reporting  requirements,  a  detailed  performance 
requirements  summary,  and  remedies  for  substandard  performance.  Without  a  clear 
understanding  regarding  metrics  being  imposed  under  the  PBSA,  the  technical  personnel 
were  reviewing  all  processes  instead  of  focusing  on  the  metrics,  which  could  have 
assisted  the  organization  in  meeting  their  GPRA  requirements  (Seven  Steps  to  PBSA, 
2005).  The  government  spent  additional  resources  gauging  the  contractor’s  performance, 
yet  it  is  unknown  how  this  related  to  the  performance  standards  and  mission  goals.  On 
the  other  hand,  both  the  government  and  contractor  interviewees  felt  that  metrics  are 
important  and  that  training  was  needed  to  assist  them  in  this  area. 
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c. 


Incentives  and  Remedies 


Besides  ensuring  the  solicitation  contains  the  appropriate  metrics,  the 
acquisition  is  also  shaped  by  the  type  of  contract  incentives.  The  research  showed  that 
contract  incentives  could  be  monetary  or  non-monetary,  as  well  as  being  positive  or 
negative  (PBSA  in  the  DoD,  2000).  It  is  interesting  to  note  that  the  survey  results  and 
interviewees  did  indicate  that  their  contract  had  negative,  monetary  incentives. 
Therefore,  the  only  remedy  to  motivate  the  contractor  for  not  meeting  an  Acceptable 
Quality  Level  (AQL)  was  to  reduce  monies  based  on  a  formula  included  in  the  contract. 
These  discrepancies  can  be  attributed  to  the  lack  of  training  and  experience  that  the 
PBSA  team  had  when  they  were  developing  the  PBSA.  Motivating  the  contractor  to 
exceed  Acceptable  Quality  Levels  (AQLs)  and  contract  expectations  is  a  new  concept  for 
the  DoD.  The  team  may  not  have  realized  that  they  should  look  at  the  complexity,  risk, 
and  dollar  value  of  the  PBSA  when  they  are  formulating  the  type  of  incentive  and/or 
remedies  to  include  during  the  development  of  the  PBSA.  They  most  likely  reviewed  the 
FAR  guidance,  which  states  that,  “Firm  Fixed  price  contracts  are  generally 
appropriate...”  and  decided  to  look  no  further  (FAR  37.602-4).  Thus,  the  team  limited 
the  government’s  ability  to  include  other  incentive  arrangements.  Consequently,  PBSA 
teams  generally  structured  their  PBSAs  based  on  firm  fixed  pricing. 

As  one  of  the  TACOM  contractor  interviewee  stated  that  under  this 
scenario,  the  contractor  focused  on  perfonning  the  services  that  provided  them  with  the 
most  profit.  Again,  both  the  contractor  and  government  personnel  stated  that  the  lack  of 
training  and  experience  could  have  attributed  to  the  way  the  incentives  were  structured  in 
(of)  the  solicitation.  Another  factor  that  was  brought  out  during  the  interviews  was  that 
an  A-76  cost  comparison  process  appears  be  inflexible  and  does  not  allow  a  teaming 
environment  and/or  the  sharing  of  ideas  throughout  the  acquisition  process.  Moreover,  if 
the  PBSA  team  would  have  reviewed  FAR  37.6  and  then  looked  at  FAR  16.103(b),  they 
may  have  realized  that  they  were  not  limited  to  structuring  the  solicitation  as  a  firm  fixed 
price  contract  (FAR  37.602-4).  Consequently,  other  contract  types  can  be  considered 
“...that  will  appropriately  tie  profit  to  contractor  performance"  (FAR  16.202). 
Nevertheless,  all  respondents  agreed  that  each  PBSA  should  be  structured  to  the  meet  the 
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requirements  of  the  services  required.  Therefore,  the  PBSA  team  should  be  aware  that 
there  is  not  a  "one  size  fits  all"  approach;  instead,  the  incentive  structure  should  be  geared 
to  the  acquisition,  the  characteristics  of  the  marketplace,  and  the  objectives  the 
government  seeks  to  achieve  (Seven  Steps  for  PBSA,  2005). 

5.  Awarding  the  Contract 

As  discussed  in  Chapter  II,  selecting  the  right  contractor  for  the  job  is  essential  in 
order  to  ensure  that  mission  goals  are  met  and  customers  are  satisfied  (Seven  Steps  to 
PBSA,  2005).  This  process  can  be  hampered  with  numerous  problems  from  improperly 
structuring  the  solicitation  to  lack  of  training  and  experience  of  the  individuals  evaluating 
offerors  proposals  (Seven  Steps  to  PBSA,  2005).  While  the  A-76  process  and  a  protest 
can  be  linked  to  the  selection  of  the  contractor  for  the  previous  IT  PBSA,  it  is  unclear  if 
the  outcome  would  have  been  different  if  the  acquisition  had  been  structured  differently. 
In  interviewing  the  contractor  personnel,  they  stated  the  A-76  cost  comparison  process 
tends  to  promote  a  hostile  environment,  no  matter  who  is  selected.  Therefore,  some 
companies  will  not  even  submit  proposals  against  the  government  when  the  A-76  process 
is  involved.  The  Government  IT  manager  interviewed  stated  that  with  the  recent  A-76 
PBSA,  there  were  numerous  changes  in  the  acquisition  personnel  assigned  to  the 
acquisition.  This  contradicts  the  intent  that  has  been  echoed  in  the  literary  review  topic 
regarding  the  PBSA  team  (Seven  Steps  to  PBSA,  2005).  The  government  technical  IT 
interviewee  stated  that  there  were  five  contracting  officers  who  were  involved  in  various 
stages  of  the  acquisition.  The  research  showed  that  the  contracting  officer  who  awarded 
the  previous  IT  PBSA  was  different  than  the  one  last  assigned  to  administer  and  monitor 
the  contract.  This  situation  did  not  help  with  the  surveillance  of  the  contract  since  the 
contracting  officer  administering  the  contract  had  no  previous  knowledge  on  how  the 
acquisition  strategy  was  developed. 

Still,  all  survey/interview  respondents  stated  that  both  the  IT  technical  and/or  the 
1102  acquisition  personnel  lacked  the  expertise  and  training  in  PBSA.  Without  the 
expertise  in  PBSAs,  the  PBSA  team  had  a  difficult  time  in  selecting  “the  right  contractor” 
(Seven  Steps  to  PBSA,  2005).  Consequently,  both  the  contractor  and  government 
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personnel  emphasized  the  importance  of  PBS  A  training  to  ensure  that  the  government 
can  select  a  contractor  who  can  provide  the  highest  quality  service  at  a  reasonable  cost 
(Seven  Steps  to  PBSA,  2005). 

6.  Contractor  Surveillance 

According  to  the  literary  review,  the  PBSA  team  should  also  be  involved  in  the 
contractor  surveillance.  Only  then  can  the  Government  ensure  that  it  is  gauging  the 
contractor’s  performance  based  on  the  objectives  stated  in  the  PWS.  As  discussed  in 
Chapter  II,  GAO  recognizes  surveillance  is  critical  to  the  organization’s  mission  (GAO 
Report  Number  05-274,  2005).  Thus,  the  individuals  conducting  surveillance  must  have 
the  knowledge  and  background  necessary  to  gauge  the  contractor’s  perfonnance  (GAO 
Report  Number  05-274,  2005).  However,  this  assumes  that  the  PWS  has  been  written  to 
ensure  that  the  organization’s  objectives  are  being  met.  Based  on  the  surveys  conducted 
for  this  project,  it  is  unclear  that  the  desired  outcomes  are  being  achieved.  As  the 
contractor  interviewee  stated,  the  Government  has  not  clearly  defined  what  their  desired 
outcomes  are  and  has  resulted  in  numerous  contract  modifications.  As  one  of  the 
TACOM  contractors  stated,  the  government  is  unaware  of  what  their  desired  outcomes 
are  and,  often,  the  contractor  has  to  assist  the  government  in  determining  their  outcomes 
after  contract  award.  Furthermore,  this  contractor  stated  that,  in  his  opinion,  performing 
surveillance  is  a  new  concept  for  the  government.  According  to  the  interviewee,  the 
Government  has  a  hard  time  conducting  surveillance  since  this  is  a  new  mindset  in  which 
the  government  has  little  or  no  experience.  Consequently,  to  overcome  this  mindset,  the 
PBSA  team  must  ask  the  following  questions:  what  are  we  measuring,  what  are  the 
desired  or  intended  outcomes,  and  how  does  this  relate  to  the  organization’s  mission 
objective  (KnowNet,  2005). 

Additionally,  the  government  technical  interviewee  stated  that  they  did  not  write 
the  QASP  for  the  previous  contract;  therefore,  they  did  not  know  if  the  QASP  clearly 
defined  the  intended  results.  This  contradicts  the  literary  guidance  which  states  that  the 
PBSA  team  should  prepare  both  the  PWS  and  the  QASP  (KnowNet,  2005).  On  the  other 
hand,  both  the  government  and  contractor  personnel  agreed  that  the  QASP  and 
surveillance  techniques  could  be  viewed  as  subjective.  As  the  literary  research  has 
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shown,  “there  is  nothing  wrong  with  the  subjective  measurement  of  a  service  as  long  as  it 
is  done  in  accordance  with  a  reasonable,  fair  and  consistently  applied  standard” 
(KnowNet,  2005). 

The  TACOM  IT  Contractor  stated  that  surveillance  should  be  done  on  a  bell- 
shape  curve  basis.  This  would  ensure  that  the  highly  visible,  critical  elements  are  running 
smoothly  at  all  times.  The  contractor  could  then  focus  their  performance  on  these 
elements  to  ensure  the  IT  infrastructure  is  being  maintained  to  avoid  downtime  and/or 
any  degradation  to  systems/equipment. 

Still,  both  the  government  and  contractor  management  interviewed  agreed  that 
surveillance  could  be  viewed  negatively.  For  example,  the  previous  IT  PBSA 
surveillance  team  focused  their  results  on  how  much  money  could  be  deducted  from  the 
contract  instead  of  the  desired  objective  of  providing  the  highest  quality  service  to  their 
customers. 

As  real  world  experiences  and  literary  review  have  shown,  the  contracting  officer 
is  often  tasked  with  developing  the  team  relationship  between  the  government  and  the 
contractor.  This  often  requires  the  Contracting  Officer  to  assume  the  role  of  an  arbitrator 
to  ensure  that  both  the  government  and  contractor  are  being  treated  equitably. 

In  the  case  of  the  most  recent  IT  PBSA,  the  warranted  Contracting  Officer  stated 
that  they  were  even  involved  in  Alternate  Dispute  Resolution  (ADR)  as  a  means  of 
resolving  differences  between  the  contractor  and  the  government  technical  personnel. 
The  ADR  approach  was  used  to  resolve  differences  between  the  contractor  and  the 
government  technical  personnel.  The  differences  ranged  from  measurable  standards,  an 
acceptable  quality  level,  incentives,  etc.  All  parties  surveyed  indicated  that  this  situation 
was  partly  attributed  to  the  fact  that  the  government  lost  the  A-76  competition. 
Consequently,  the  surveillance  team  was  perceived  to  be  too  harsh  and  did  not  always 
appear  to  conduct  themselves  with  the  utmost  integrity. 

On  the  other  hand,  the  contractor  personnel  stated  that,  in  their  opinion,  the 
QASPs  are  not  necessarily  a  good  tool  to  gauge  the  contractor’s  performance.  They  felt 
that  the  same  surveillance  could  be  accomplished  by  scheduling  weekly,  biweekly, 
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monthly,  or  quarterly  In-process  Review  (IPR)  meetings.  These  IPRs  can  be  structured 
with  an  agenda  and  can  provide  a  forum  for  both  the  government  and  the  contractor  to 
gauge  if  the  intended  outcomes  are  being  accomplished  and,  if  not,  why.  Additionally, 
these  IPRs  can  be  used  as  a  forum  in  establishing  and  building  the  relationship  between 
the  two  parties. 

Consequently,  it  is  imperative  that  the  team  be  established  from  the  onset,  and  the 
team  should  consist  of  technical  personnel,  which  will  eventually  have  the  responsibility 
to  perform  contractor  surveillance.  When  a  team  member  has  to  be  replaced,  the  new 
team  member  should  receive  PBS  A  training  prior  to  joining  the  team.  The  team  member 
should  possess  technical  “knowledge,  experience,  and  insight  into  what  needs  to  happen 
next,  and  what  is  expected  during  contract  perfonnance.  Contract  award  is  not  the  final 
measure  of  success.  Effective  and  efficient  contract  perfonnance  that  delivers  a  solution 
is  the  goal.  The  team  should  stay  together  to  see  that  end  reached”(Seven  steps  to  PBSA, 
2005). 

C.  SUMMARY 

In  conclusion,  this  chapter  provided  a  comparison  and  analysis  of  the  literary 
review  discussed  in  Chapter  II,  with  the  surveys  and  interviews  discussed  in  Chapter  III. 
The  research  showed  that  the  government  personnel  lack  the  training  and  expertise 
needed  in  order  to  clearly  define  their  requirements  in  order  to  perform  adequate 
contractor  surveillance.  Additionally,  survey  results  show  that  PBSA  members  need  to 
have  an  understanding  of  their  roles  and  responsibilities  in  any  IT  PBSA  (Seven  Steps  to 
PBSA,  2005).  The  surveys  indicated  that  technical  personnel  felt  that  acquisition 
personnel  lack  knowledge  in  the  IT  process,  which  can  hinder  their  ability  to  recommend 
the  correct  acquisition  strategy.  Additionally,  technical  personnel’s  lack  of  knowledge 
regarding  the  acquisition  process  can  cause  problems  when  it  comes  to  contractor 
surveillance.  Therefore,  technical  personnel  who  will  be  responsible  for  monitoring  the 
contractor’s  perfonnance  need  to  understand  their  responsibilities  long  before  the 
contract  is  awarded  (Seven  Steps  to  PBSA,  2005).  Furthermore,  Chapter  V  will  provide 
the  research  conclusions,  recommendations,  and  areas  for  further  research. 


56 


V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

As  stated  in  Chapter  I,  the  purpose  of  this  Joint  Applied  Project  professional 
report  is  to  analyze  IT  performance  based  service  acquisitions  within  the  U.S.  Army  Tank 
Automotive  and  Armaments  Command  (TACOM)  and  to  evaluate  what  performance 
measurements  are  being  used  to  gauge  the  contractor’s  effectiveness  against  the 
performance  measures  stated  in  the  contract.  In  addition,  this  research  will  evaluate 
incentives,  as  well  as  penalties  used,  to  incentivize  the  contractor  to  provide  the  level  of 
service  envisioned  by  the  Government. 

In  conclusion,  the  purpose  of  this  research  will  analyze  how  the  development  of 
performance  measures,  incentives,  and  penalties  affect  the  ability  of  the  surveillance  team 
to  adequately  administer  the  contract.  This  information  will  be  used  to  develop  and  post 
a  perfonnance  measurement  guide  on  the  TACOM  Community  website  to  provide  all 
contracting  personnel  with  means  to  develop  performance  measures  for  IT  PBSAs. 

In  order  to  accomplish  this  research,  the  following  questions  were  developed: 

1.  To  what  extent  are  performance-based  measures  being  used  to  manage 
TACOM  IT  PBSAs? 

2.  How  clear  and  applicable  are  PBSA  contracting  policies  and  procedures, 
e.g.,  statements  of  work  (SOWs),  performance  work  statements  (PWS),  and  statements  of 
objectives  (SOOs)? 

3.  What  are  the  incentives  and  penalties  affecting  PBS  contracting? 

4.  What  are  ways  to  improve  U.S.  Army,  TACOM-RI,  PBSAs,  including 
methods  promulgating  web-based  guidance  to  practitioners? 

This  chapter  will  discuss  the  research  questions  and  provide  conclusions, 
recommendations,  and  areas  for  further  research  based  on  the  analysis  performed  during 
this  research. 
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B.  CONCLUSIONS 

In  response  to  the  first  question  evaluated  under  this  research  which  asked,  “To 
what  extent  are  performance-based  measures  being  used  to  manage  TACOM  IT  PBSAs,” 
the  research  indicated  that  nearly  all  the  acquisitions  contained  performance  measures. 
However,  the  research  showed  that  the  performance  measures  did  not  necessarily  yield 
the  desired  outcomes.  In  addition,  the  research  of  both  governmental  guidance  and  best 
commercial  practices  showed  that  these  perfonnance  measures  should  be  kept  to  a 
minimum  in  order  to  adequately  perform  contract  surveillance. 

However,  in  reality,  the  research  showed  that  often  the  PWS,  SOO,  QASP,  etc., 
included  too  many  performance  standards.  This  contributed  to  the  lack  of  incentives  to 
motivate  the  contractor  to  exceed  expectations,  especially  on  firm  fixed  price  type 
contracts.  The  research  showed  that  contractors  would  often  zero  in  on  the  areas  where 
they  can  gain  the  most  profit  at  the  expense  of  other  tasks  required  under  the  contract. 
Although  the  perfonnance  measures  are  included  in  the  PBSA,  they  are  not  necessary 
yielding  the  desired  outcomes.  This  can  be  attributed  to  the  PBSA  team’s  lack  of 
experience  and  training  regarding  PBSA. 

Additionally,  the  literary  review  on  PBSA,  PWS,  and  QASP  indicates  that 
performance  measures  should  begin  with  the  definition  of  the  requirement;  the  research 
showed  this  was  not  always  being  done.  This  cannot  be  accomplished  unless  the  PBSA 
team  has  a  clear  definition  of  the  requirements.  To  accomplish  this  task,  the  PBSA  team 
must  continuously  work  with  the  stakeholders,  i.e.,  customer(s)  to  ensure  that  they  have  a 
clear  understanding  of  the  desired  outcomes.  Consequently,  it  is  essential  that  the  team 
members  remain  consistent  throughout  the  PBSA  (Seven  Steps  to  PBSA,  2005). 

In  response  to  the  second  question,  the  objective  of  this  research  was  to  find  out  if 
the  current  policies  and  procedures  regarding  PBSAs  are  clearly  identified  and  applicable 
for  this  type  of  acquisition  technique.  This  includes  documents  such  as  statements  of 
work  (SOWs),  performance  work  statements  (PWS),  and  statements  of  objectives 
(SOOs). 
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The  literary  review  conducted  in  Chapter  II  indicates  that  there  are  many  PBS  A 
guidebooks  and  reference  sites  which  can  be  found  on  the  internet.  However,  as  part  of 
the  research  into  PBSA,  it  was  discovered  that  there  is  some  confusion  regarding  what 
topic  to  look  for,  i.e.,  is  the  acquisition  under  PBC,  PBSC,  and/or  PBSA.  In  fact,  the 
OFPP  Interagency  Task  Force  has  issued  recommendations  to  clear  up  this  confusion. 
The  recommendations  include  adding  definitions  to  FAR  Part  2  for  PWS,  QASP,  SOO, 
etc.  In  addition,  this  report  rescinded  out  of  date  guidance  and  the  OFPP  Best  Practice 
guide  of  1998  in  favor  of  the  Seven  Steps  to  PBSA  guide  (PBSA  Contracting  for  the 
Future,  2003). 

As  discussed  in  Chapter  II  -  Literary  Review,  this  guide  provides  seven 
comprehensive  steps  along  with  references  in  performing  any  PBSA.  Furthermore,  the 
KnowNet  website  at  http://knownet.hhs.gov/aboutKnowNet.htm  provides  a  handy 
desktop  reference  and  is  broken  out  into  the  various  acquisition  phases,  beginning  with 
the  formulation  of  the  PBSA  team  through  contract  completion  and  close  out. 

Another  resource  is  the  Department  of  Defense  Education  Activity  (DoDEA) 
customer  guide,  which  provides  non-acquisition  personnel  with  guidance  to  Federal 
acquisition  systems  (DoDEA,  2005).  Consequently,  there  are  plenty  of  reference 
materials  available  to  assist  the  PBSA  team  with  any  PBSA.  Even  though  there  is 
guidance  on  PBSA,  each  team  member  should  have  PBSA  training.  This  will  assist  the 
team  in  being  able  to  evaluate  and  narrow  down  their  guidance  material  specifically  for 
their  particular  type  of  acquisition. 

The  third  question  used  in  this  research  project  investigated  the  incentives  and 
penalties  affecting  PBS  contracting  and  found  that  there  are  many  types  of  incentives 
available,  both  non-monetary  and  monetary.  The  research  showed  that  often  times,  the 
PBSA  team  reviews  the  context  at  FAR  37.6  and  uses  firm  fixed  pricing  as  a  technique  to 
incentivize  the  contractor  to  perform  the  required  services.  Based  on  the  surveys, 
interviews,  and  literary  review,  this  can  be  attributed  to  the  PBSA  team’s  lack  of 
knowledge  and  experience  in  PBSA.  Additionally,  the  team  may  lack  the  appropriate 
training  to  be  able  to  identify  what  added  value  incentives  can  bring  in  contractor 
surveillance. 
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To  conclude,  the  fourth  question  used  in  this  research  project  was  previously 
addressed  in  Chapter  IV.  The  project  reviewed  the  information  provided  under  Chapter 
IV,  Analysis  and  Findings,  which  indicated  that  there  are  various  disconnects  and 
discrepancies  that  need  to  be  addressed  in  any  PBSA.  Chapter  IV  also  provided  various 
ways  that  a  team  can  improve  U.S.  Army  and  TACOM-RI  PBSAs,  to  include  methods 
promulgating  web-based  guidance  to  the  entire  acquisition  workforce.  These  methods 
include  the  following: 

1.  Establishing  the  PBSA  Team 

A  literary  review  of  Commercial  Best  Practices  and  government  sources  has 
indicated  that  PBSA  teams  should  be  formed  early  in  the  acquisition  process.  The  real 
world  infonnation  obtained  in  Chapter  III  indicates  this  is  not  what  is  currently 
happening  in  TACOM  acquisitions.  In  addition,  the  research  indicated  that  there  is  a  lack 
of  consistency  in  team  membership,  which  is  inconsistent  with  the  principles  stated  in  the 
Seven  Steps  of  PBSA. 

The  team  must  have  senior  management  support  to  ensure  any  PBSA  is 
successful.  This  support  includes  a  commitment  that  the  PBSA  team  who  develops  the 
PWS  is  that  same  team  that  provides  surveillance  after  contract  award. 

Additionally,  the  research  shows  that  many  of  the  PBSA  team  members  have  not 
received  sufficient  training  on  PBSAs  to  be  an  effective  team  member.  Yet,  there  is  no 
indication  that  training  has  been  a  requirement  in  the  past  or  will  be  in  the  future  for 
PBSA  team  membership.  To  be  an  effective  member  of  the  PBSA  team,  all  team 
members  should  have  documented  proof  that  they  have  received  training  in  the  PBSA 
process  and  have  a  basic  understanding  of  the  acquisition  process. 

Furthermore,  the  research  has  shown  that  the  acquisition  workforce  does  not 
necessarily  have  the  experience  or  the  knowledge  base  required  to  perform  a  PBSA. 
Therefore,  the  acquisition  workforce  must  receive  training  on  PBSAs  and,  if  possible,  be 
allowed  to  participate  on  a  PBSA  before  they  are  assigned  the  responsibility  of  a  PBSA. 
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The  TACOM  organization  must  recognize  and  support  the  PBSA  team  in  the  challenges 
they  may  face  throughout  the  PBSA  process.  Only  then  will  the  team  feel  empowered 
enough  to  possibly  think  outside  the  box. 

2.  Defining  the  Requirement 

The  research  indicates  that  more  attention  needs  to  be  given  to  defining  the 
requirement.  The  team  should  assist  the  customer  in  identifying  their  requirements  in 
terms  of  outcomes  instead  of  stating  how  to  perform  the  services  required. 

Identification  of  the  requirement  should  be  associated  with  the  organization’s 
mission  and  must  be  measurable.  However,  in  order  for  this  to  occur,  the  PBSA  team 
and  the  organization  must  move  away  from  doing  business  as  usual.  This  will  involve 
significant  changes  in  the  way  TACOM  currently  conducts  their  PBSAs,  and  they  cannot 
be  successful  without  conducting  adequate  market  research. 

3.  Conducting  Market  Research 

All  acquisitions,  including  PBSAs,  are  required  to  have  documentation  that  shows 
adequate  market  research  was  conducted.  Yet,  many  in  the  acquisition  workforce  and 
PBSA  team  members  do  not  know  what  constitutes  adequate  market  research  or  how  to 
go  about  it. 

Acquisition  personnel  are  usually  brought  into  the  PBSA  acquisition  process  after 
the  requirement  has  been  identified.  As  a  result,  market  research  is  usually  performed 
after  the  customer’s  minimum  needs  have  been  recognized. 

The  responsibility  of  conducting  market  research  was  designed  to  be 
accomplished  by  the  Program  Manager  as  they  have  the  market  expertise  to  conduct 
adequate  market  research;  but  the  requirement  to  conduct  the  market  research  has  been 
passed  to  the  contracting  officers/  contract  specialist  (1102  s). 

Traditionally,  the  1 102s  do  not  have  the  market  expertise  nor  the  background  and 
experience  to  be  market  analysts.  As  a  result,  1102s  are  now  expected  to  assume  a 
different  role  and  become  market  experts.  Until  1 102s  are  able  to  get  more  experience  in 
becoming  experts  in  the  markets  of  the  services  they  purchase,  adequate  market  research 
will  not  be  completed. 
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By  conducting  adequate  market  research,  the  PBSA  team  will  be  able  to 
determine  the  appropriate  contract  type,  terms  and  conditions,  length  of  the  contract, 
incentives,  and  industry  practice  and  standards. 

Without  adequate  market  research,  the  best  solutions  available  to  meet  the 
customer  requirements  may  not  be  considered.  This  results  in  the  customer  not  being 
satisfied  and  the  organizations  mission  not  being  met. 

The  Seven  Steps  to  PBSA  states  the  team  should  use  commercial  standards 
whenever  possible,  which  could  assist  the  team  in  formulating  their  perfonnance 
standards  and  metrics.  Additionally,  if  there  are  industry  standards  available,  then  all 
potential  offerors  should  already  know  what  the  expected  performance  is  in  any  resulting 
contract. 

In  addition,  market  research  can  assist  the  team  in  selection  of  contract  type  and 
incentives  for  the  PBSA. 

Still  the  team  and  the  organization  should  place  more  emphasis  on  perfonning 
market  research  throughout  the  process  (Seven  Steps  to  PBSAs,  2005). 

4.  Solicitation  Development  -  SOW,  Metrics,  Incentives,  etc. 

It  is  essential  that  the  PBSA  team  write  the  PWS  in  relevant  and  measurable 
terms.  Additionally,  the  PWS  must  be  written  in  terms  of  what  the  desired  outcomes  are 
and  not  what  or  how  the  services  were  previously  performed.  This  is  a  cultural  change 
for  the  technical  personnel  who  once  provided  the  services.  Often  the  COR,  COTR,  or 
the  QAE  previously  performed  the  same  function  and  are  now  responsible  for  monitoring 
the  contractor  performance.  Consequently,  it  is  imperative  that  PBSA  team  applies  sound 
business  judgment  when  writing  the  PWS  to  ensure  that  the  government  provides  all 
offerors  with  clear,  concise,  and  measurable  standards  for  the  required  services. 
Otherwise,  contractors  may  fail  to  submit  alternative  approaches  to  perform  the  same 
services,  which  could  result  in  a  lower  cost  to  the  customer  or  a  higher-level  of  service. 

5.  Measurable  Performance  Standards  or  Metrics 

Although  the  literary  review  on  PBSA  provided  information  and  references 
regarding  metrics,  it  is  unclear  that  the  Government  personnel  surveyed  had  any 
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knowledge  or  experience  in  selecting  and  evaluating  metrics  for  any  PBSA.  The  research 
indicated  that  the  PBSA  team  should  use  existing  commercial  quality  standards,  if 
applicable.  These  standards  should  be  identified  during  market  research  if  done  properly 
(Seven  Steps  to  PBSA,  2005). 

Additionally,  a  Quality  Assurance  Plan  (QAP)  should  be  requested  as  part  of  the 
acquisition  process.  It  ensures  that  the  government  receives  the  required  services  under 
any  resulting  contract.  This  plan  provides  detailed  steps,  which  the  contractor  must  take 
to  make  sure  that  they  perform  in  accordance  with  the  performance  standards  as  stated  in 
the  contract. 

The  QAP  sets  the  "what,"  "how,"  and  "how  often"  of  quality  assurance,  and  helps 
the  offeror  determine  the  level  of  resources  needed  to  adequately  staff  the  project.  It  also 
reflects  the  value  of  each  performance  requirement  as  a  percentage  of  the  overall 
contract,  so  the  offeror  can  focus  his  resources  in  the  appropriate  areas”  (Rogin,  2001). 

However,  it  does  not  have  to  be  made  part  of  the  resulting  award.  Usually  after 
award,  the  contractor  will  submit  a  quality  control  plan  (QCP).  The  QCP  must  be 
approved  soon  after  the  contract  is  awarded  and  is  often  modified  as  needed  to  cover 
unanticipated  changes  or  new  technologies.  As  a  result,  the  PBSA  team  must  conduct 
adequate  market  research  to  assist  them  in  identifying  the  appropriate  metrics  needed  for 
each  PBSA. 

6.  Incentives  and  Remedies 

The  literary  review  indicated  that  incentives  could  be  monetary  or  non-monetary 
and  negative  or  positive.  However,  it  is  unclear  when  or  what  incentives  are  appropriate 
for  each  PBSA  reviewed.  The  incentives  are  used  to  motivate  contractor  performance. 
Many  of  the  acquisitions  that  were  reviewed  during  this  research  project  were  Finn  Fixed 
Price  (FFP).  Often  the  FFP  contracts  are  viewed  to  have  a  negative  incentive,  and  the 
focus  is  on  deductions  for  non-performance  instead  of  motivating  the  contractor  to 
exceed  contract  expectations.  Consequently,  the  PBSA  team  and  acquisition  personnel 
have  not  taken  into  consideration  the  value  added  importance  that  incentives  can  bring  in 
contractor  surveillance. 
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7.  Awarding  the  Contract 

As  stated  in  the  Seven  Steps  to  PBSA,  the  “  key  to  selecting  the  right  contractor  is 
to  structure  the  acquisition  so  that  the  government  describes  the  problem  that  needs  to  be 
solved  and  vendors  compete  by  proposing  solutions.”  This  will  assist  the  PBSA  team  in 
selecting  a  contractor  who  offers  a  quality  solution  to  the  PWS  and  the  contractor’s 
proposal  will  then  “becomes  a  true  discriminator  in  best-value  evaluation”  (Seven  Steps 
to  PBSA,  2005). 

However,  the  research  of  the  real  world  experiences  indicates  that  best  value 
acquisitions  often  are  selected  by  the  lowest  price  offered.  In  conclusion,  the  PBSA  team 
needs  to  evaluate  the  PBSA  to  see  if  best  value  techniques  are  appropriate  and,  if  so,  that 
they  have  the  knowledge  and  experience  to  select  the  appropriate  evaluation  factors 
which  will  yield  the  best  value  to  the  government.  Thus,  if  the  contractor  represents  the 
best  value  to  the  government,  then  the  team  should  be  able  to  monitor  the  contractor’s 
performance  based  on  the  infonnation  provided  during  the  solicitation. 

8.  Contract  Surveillance 

As  GAO  report  05-274  stated,  DoD  has  not  adequately  performed  contractor 
surveillance  (GAO  report  05-274,  2005).  Yet  this  is  not  the  case  based  on  the  results 
from  survey  and  interviews  performed  for  this  research  project.  The  QAEs  perform 
contractor  surveillance  either  on  a  daily,  weekly  or  monthly  basis. 

The  research  also  indicated  that  the  QAEs  often  sought  ways  to  reduce  the 
contract  amount  due  to  inadequate  and/non-performance  issues.  The  QASP  contained 
too  many  tasks  that  were  being  measured;  however,  they  might  have  been  relevant  to  the 
overall  performance  of  the  contract,  or  they  may  have  decreased  the  significance  of  each 
task  to  where  the  contractor  picks  and  chooses  which  task  to  perform. 

Consequently,  the  QAEs  and  the  PBSA  team  need  to  remember  the  importance  of 
obtaining  desired  outcomes  and  meeting  organizational  mission  and  goals  when  writing 
the  PWS  and  the  QASP.  This  will  ensure  that  customer’s  needs  are  being  met. 
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c. 


RECOMMENDATIONS 


In  order  to  be  able  to  increase  the  number  of  PBSAs  being  done  in  the 
organization  and  to  ensure  the  ability  of  the  PBS  A  team  to  effectively  perform 
surveillance  on  the  resulting  contracts,  the  research  indicates  that  DoD’s  culture  must 
change.  The  workforce,  including  management,  must  have  a  clear  focus  on  the  reasons 
why  PBSAs  are  important  and  how  they  can  help  the  PBSA  team  ensure  that  the  PBSA 
will  meet  the  performance  goals  in  order  to  accomplish  the  organization’s  overall 
mission. 

Additionally,  the  PBSA  team  must  let  the  contractor  provide  solutions  to  the 
stated  problem  and  let  the  contractor  be  responsible  for  contract  performance.  If  the 
government  directs  the  contractor  on  how  to  perfonn  the  services,  then,  in  essence, 
“they've  taken  responsibility  away  from  the  contractor  and  onto  themselves,"  (Mathers, 
2001). 

To  help  ensure  contractor  provides  the  level  of  performance  that  the  government 
has  envisioned,  the  contractor  must  become  a  part  of  the  team  after  contract  award  and 
must  be  treated  as  such.  This  is  another  cultural  change  for  the  government  acquisition 
workforce. 

Consequently,  the  team  takes  on  a  different  role  of  working  with  the  contractor 
and  also  monitoring  their  perfonnance.  The  key  is  to  build  on  the  relationship  with  the 
contractor  and  to  find  ways  to  motivate  them  to  exceed  expectations;  thus,  satisfying  the 
customers  needs  while  meeting  the  organizational  goals. 

Consequently,  it  is  important  the  PBSA  team  be  established  from  the  onset  of  the 
acquisition  and  remains  consistent  throughout  the  acquisition  process  to  contract 
closeout. 

1.  Team  Consistency 

As  identified  in  our  literary  review  regarding  PBSA,  both  industry  and 
government  resources  have  stated  that  in  order  to  adequately  perform  contractor 
surveillance  there  must  be  consistency  in  the  team  membership.  Organizations  must 
ensure  that  the  team  who  initiates  the  acquisition  should  also  be  the  team  that  performs 
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contractor  surveillance.  After  all,  they  are  the  ones  that  have  the  knowledge  to  judge 
whether  the  performance  standards  are  being  met  or  if  changes  are  in  order. 

2.  Conduct  Adequate  Market  Research 

The  PBSA  team  needs  to  be  held  accountable  for  conducting  adequate  market 
research.  They  need  to  understand  that  market  research  is  not  a  one-time  effort  and  not 
the  responsibility  of  the  contracting  officer.  Additionally,  they  need  to  perform  market 
research  in  conjunction  with  assisting  the  customer(s)  in  identifying  their  minimum 
needs.  Therefore,  open  communication  is  required  with  all  stakeholders  involved  in  the 
PBSA. 

The  literary  review  states  that  the  team  can  obtain  assistance  from  private  industry 
regarding  market  research.  However,  this  does  not  solve  the  problem  of  knowing  when 
and  how  to  perfonn  market  research.  Market  research  is  just  not  the  identification  of 
solutions  for  the  problem;  it  also  includes  terms  and  conditions,  length  of  the  contract, 
incentives,  industry  practices  and  standards. 

The  Seven  Steps  to  PBSA  states,  the  team  should  use  commercial  standards 
whenever  possible,  which  could  assist  the  team  in  formulating  their  perfonnance 
standards  and  metrics.  Additionally,  if  there  are  industry  standards  available,  then  all 
potential  offerors  should  already  know  what  the  expected  performance  is  in  any  resulting 
contract. 

In  addition,  market  research  can  assist  the  team  in  selection  of  contract  type  and 
incentives  for  the  PBSA.  The  PBSAs  reviewed  at  TACOM  were  firm  fixed  price  (FFP) 
type  contracts.  These  types  of  contracts  place  the  total  risk  on  the  contractor  and  should 
incentivize  them  to  hold  down  costs.  However,  this  type  of  vehicle  may  not  be  the 
correct  vehicle  to  use  on  PBSA.  The  team  needs  to  make  sure  that  the  acquisition 
contains  incentives  which  will  allow  the  contractor  to  propose  innovative  approaches  and 
ultimately  provide  savings  to  the  government.  Consequently,  the  incentives  included  in 
the  acquisition  must  be  equivalent  to  the  risk  and  complexity  of  the  PBSA. 
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3.  Knowledge  of  Monitoring  Tools 

The  literary  review  on  PBSA  revealed  that  there  are  no  templates  for  PWS, 
QASP,  etc.  (Guideline  to  PBSA,  2001).  Still,  the  PBSA  team  should  be  able  to  obtain 
lessons  learned  and  copies  from  present  or  past  acquisitions  to  assist  them  in  drafting 
these  documents  from  various  resources,  such  as,  KnowNet,  Seven  Steps  to  PBSA,  etc. 
This  infonnation  can  help  the  COR,  COTR,  and/or  the  QAE  with  the  use  of  surveillance 
tools  such  as  QASP  and  Acceptable  Quality  Level  (AQL).  After  all  they  are  responsible 
for  ensuring  that  the  contractor  is  meeting  the  desired  outcomes. 

The  QASP  is  an  essential  piece  of  any  PBSA.  Yet  there  is  no  clear  and  concise 
guidance  on  how  to  develop  one.  In  order  to  achieve  this,  the  PBSA  team  should  make 
sure  that  the  QASP  reflects  the  contract  objectives.  In  addition,  the  team  needs  to  make 
certain  that  the  level  of  surveillance  reflects  the  critical  level,  risk,  dollar  value,  etc.,  of 
the  required  services  (Acquisition  Advisory  Panel  Performance-Based  Contracting 
Subpanel  Report,  2005). 

The  DOE  Acquisition  guidebook  states  “The  QASP  concentrates  on  the  level  of 
performance  and  not  the  method  for  achieving  it”  (DODEA,  Customer  Guide,  2005). 

4.  Requiring  Training  on  PBSAs 

All  of  the  PBSA  team  members  should  receive  training  before  they  can  participate 
in  any  PBSA.  The  team  leader  should  have  documented  proof  for  each  team  member  of 
the  training  they  have  received. 

There  are  several  excellent  training  sources  available,  through  the  Defense 
Acquisition  University.  Training  can  often  be  brought  on-site  if  the  requirement  exists  or 
the  person  can  be  sent  to  a  training  facility.  In  addition,  there  are  several  excellent  on  line 
web  sites,  which  can  assist  the  team  with  the  PBSA.  The  KnowNet  is  comprised  of 
different  modules  and  provides  frequently  asked  questions,  along  with  reference  material. 
Also,  the  Seven  Steps  to  PBSA  provides  an  extensive  and  comprehensive  guidebook 
regarding  PBSAs.  Additionally,  the  Guide  to  PBSA,  dated  2000,  provides  the  guiding 
principles  required  for  any  PBSA.  Consequently,  the  resources  available  to  the  team  are 
abundant;  it  just  takes  time  to  research  and  time  is  not  always  available. 
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D.  AREAS  FOR  FURTHER  RESEARCH 

1.  Does  the  A-76  acquisition  process  hinder  competition  and  cooperation 
on  PBSAs? 

Although  the  A-76  acquisition  process  has  been  streamlined,  it  is  still  very 
confusing. 

2.  Is  the  Government  effectively  using  Quality  Assurance  Surveillance 
Plan  (QASP)  on  PBSA  and  are  the  plans  being  effectively  monitored? 

In  testimony  before  the  Panel,  Ms.  Barbara  S.  Kinosky  from  Centre  Consulting 
stated,  “Government  needs  to  learn  not  to  create  overly  burdensome  surveillance  plans 
that  will  ultimately  create  a  bureaucracy  of  contractors,  monitoring  contractors  for 
compliance,  only  evaluate  what  is  necessary  to  accurately  measure  success”  (Ensuring 
the  Effective  and  Appropriate  Use  of  PBC.  2005). 

3.  Is  the  traditional  1102s  receiving  adequate  training  to  assist  them  in 
performing  PBSA?  Should  their  expertise  become  more  like  that  of  a  Market 
Analyst  instead  of  just  a  manager  of  the  acquisition  process?  What  training  can  be 
provided  to  teach  our  1102s  to  become  market  analysts? 

Most  service  acquisitions  do  not  usually  have  a  specific  Program  Manager; 
therefore  it  is  usually  the  contracting  officer  responsibility  to  lead  the  PBSA.  This  role  is 
new  to  the  traditional  1102,  and  they  may  not  necessarily  possess  the  knowledge  and/or 
training  to  perform  these  duties.  Traditionally,  the  contracting  officer  or  1102  has  not  had 
to  be  a  market  analyst,  but  with  requirements  of  a  PBSA,  they  must  become  market 
analysts  and  become  technically  familiar  with  the  type  of  services  required.  In  fact,  this 
dilemma  has  lawmakers  so  concerned  that  the  recently  approved  National  Defense 
Authorization  Act  for  Fiscal  2006  provides  “for  DoD  officials  to  increase  the  number  of 
acquisition  employees  by  15  percent  by  fiscal  2009.”  Furthermore,  the  bill  also  calls  for 
“a  focus  on  hiring  workers  with  skills  in  specific  areas,  such  as  writing  perfonnance 
based  statements  of  work. . .”  (Miller,  2005).  Consequently,  the  current  workforce  should 
also  be  trained  to  perform  these  same  duties  and  to  act  more  like  a  market  analyst. 

4.  How  can  the  Government  ensure  that  the  team  remains  consistent 
throughout  the  acquisition  process? 

The  FAR  identifies  the  roles  and  responsibilities  of  the  PBSA  team,  as  well  as 
stating  that  the  team  should  be  empowered  to  make  decisions.  However,  it  fails  to  state 
that  the  team  members  should  be  committed  to  the  acquisition  even  after  contract  award. 
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As  the  research  indicated,  this  is  crucial  for  the  PBSA  team  to  be  able  to  effectively 
monitor  contractor  performance.  Yet,  how  can  the  government  ensure  that  the  team 
remains  consistent? 

Another  question  that  arises  is  how  can  we  ensure  that  the  acquisition  team 
adequately  perfonns  market  research? 

Although  the  FAR  requires  that  each  acquisition  include  adequate  market 
research,  it  is  still  being  left  to  the  acquisition  team.  Team  members  are  often  not 
familiar  with  market  research  techniques;  therefore,  they  are  not  sure  how  they  can  assist 
the  acquisition  personnel  in  perfonning  market  research. 

E.  SUMMARY/CONCLUSION 

This  chapter  concludes  the  research  perfonned  regarding  surveillance  of  PBSAs. 
As  previously  stated  in  this  chapter,  the  research  provided  a  summary  of  the  conclusions 
uncovered  during  the  research  project.  Additionally,  it  included  recommendations  which 
can  assist  the  acquisition  workforce  and  the  PBSA  team  with  all  different  stages  of  a 
PBSA.  Finally,  this  chapter  also  included  areas  for  further  research  regarding  PBSAs. 
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